Compare commits

..

46 Commits

Author SHA1 Message Date
Thomas Lamprecht a67874b6ae bump version to 1.1.14-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 18:08:02 +02:00
Thomas Lamprecht 9402e9f357 cargo: update proxmox-acme-rs to 0.3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 18:05:19 +02:00
Thomas Lamprecht b75bb5434e d/control.in: update
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 18:05:06 +02:00
Thomas Lamprecht ec44c3113b backport "datastore: lookup: reuse ChunkStore on stale datastore re-open"
Backport of commit 0bd9c87010

When re-opening a datastore due to the cached entry being stale
(only on verify-new config change). On datastore open the chunk store
was also re-opened, which in turn creates a new ProcessLocker,
loosing any existing shared lock which can cause conflicts between
long running (24h+) backups  and GC.

To fix this, reuse the existing ChunkStore, and thus  its
ProcessLocker, when creating a up-to-date datastore instance on
lookup, since only the datastore config should be reloaded. This is
fine as the ChunkStore path is not updatable over our API.

Note that this is a precaution backport, the underlying issue this
fixes is relatively unlikely to cause any trouble in the 1.x branch
due to not often re-opening the datastore.

Originally-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 18:00:01 +02:00
Thomas Lamprecht cb21bf7454 ui: add notice for nearing PBS 1.1 End-of-Life
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 17:35:03 +02:00
Dominik Csapak a1cffef503 pbs-tools: LruCache: implement Drop
this fixes the leaked memory for the cache, as we had only pointers
in the map/list which were freed, not the underlying chunks

moves the 'clear' implementation out of the trait bounds so that
Drop can reuse it

this is used e.g. for file download from a pxar

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
(cherry picked from commit 98983a9dab)
2022-01-20 15:46:35 +01:00
Wolfgang Bumiller 9b00099ead drop RawWaker usage
this was also leaking a refcount before, this is fixed now

See-also: proxmox/proxmox-async:
  * d0a3e38006fe ("drop RawWaker usage")
  * ff132e93c6fd ("rustfmt")

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-01-20 15:41:00 +01:00
Thomas Lamprecht d2351f1a81 bump version to 1.1.13-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-19 10:21:23 +02:00
Thomas Lamprecht 869e4601b4 api daemons: fix sending log-reopen command
send_command serializes everything so it cannot be used to send a
raw, optimized command. Normally that means we get an error like
> 'unable to parse parameters (expected json object)'
when used that way.

Switch over to send_raw_command which does not re-serializes the
command.

Fixes: 45b8a032 ("refactor send_command")
Originally-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-11 14:56:30 +02:00
Thomas Lamprecht 238e5b573e buildsys: prune-sim is not generated, do not cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-26 16:41:34 +02:00
Thomas Lamprecht 996680a336 bump version to 1.1.13-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-26 16:40:37 +02:00
Thomas Lamprecht 94f6127711 Revert "auth: 'crypt' is not thread safe"
With this I'm getting coredumps on every log in:

> Process 20957 (proxmox-backup-) of user 34 dumped core.
>
> Stack trace of thread 20987:
> #0  0x0000563dec9ac37f _ZN3std3sys4unix14stack_overflow3imp14signal_handler17ha95ed06a038ca319E.llvm.11547235952357801165 (proxmox-backup-proxy)
> #1  0x00007f2638de9840 __restore_rt (libc.so.6)
> #2  0x00007f2638e51dac __stpncpy_sse2_unaligned (libc.so.6)
> #3  0x00007f26393b1340 __sha256_crypt_r (libcrypt.so.1)
> #4  0x00007f26393b0553 __crypt_r (libcrypt.so.1)
> #5  0x0000563dec6e44df _ZN14proxmox_backup4auth5crypt17hd5165f960093dfe7E (proxmox-backup-proxy)

This reverts commit acefa2bb6e.
2021-07-26 16:38:16 +02:00
Thomas Lamprecht 3841301ee9 d/control: update generated build-deps
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-23 12:36:36 +02:00
Thomas Lamprecht f406202825 bump version to 1.1.13-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-23 12:35:06 +02:00
Stefan Reiter ba50f57e93 file-restore: increase lock timeout on QEMU map
This lock is held during VM startup, so that multiple calls will not
start VMs twice. But this means that the timeout needs to incorporate
the time it might take a VM to boot, so increase it quite a bit.

This could previously lead to "interrupted system call" errors when
accessing backups with many disks.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
(cherry picked from commit 66501529a2)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-23 12:30:09 +02:00
Thomas Lamprecht 61a758f67d build.rs: tell cargo to only rerun build.rs step if .git/HEAD changes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 12:43:19 +02:00
Thomas Lamprecht 847c27fbee build.rs: factor out getting git command output into helper fn
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 12:43:19 +02:00
Thomas Lamprecht 7d79f3d5f7 file restore daemon: log about basic steps
to make the log more useful..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit 9a06eb1618)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:08:15 +02:00
Thomas Lamprecht fa3fdea590 file restore daemon: reword warning about manual execution
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit 309e14ebb7)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:08:10 +02:00
Thomas Lamprecht aa2cd76c58 restore daemon: use millisecond log resolution
During startup most of the stuff is happening in milliseconds (or
less), so the timestamp granularity of seconds made it hard to tell
if the previous command required 990ms or 1ms, which is quite the
difference in the restore daemon context.

Using micros seems not to bring too much additional information, a
millisecond is already an ok lower time resolution for logging, so
switch only to millis for now.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit ecd66ecaf6)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:08:00 +02:00
Thomas Lamprecht e2d82c7d4d restore daemon: create /run/proxmox-backup on startup
fixes file restore again.

The new Memcom tracking file lives in `/run/proxmox-backup` and is
always created on REST interaction, as CachedUserInfo uses it to
efficiently track config changes, and such a cache is used in each
REST handle_request.

Further, the Memcom infra expects the base run PBS dir to exists
already, which is an OK assumption to have, but in the file-restore
daemon we have a significantly more minimal environment, and the run
dir was simply not required there, even /run isn't a tmpfs yet.

Fixes fda19dcc6f ("fix CachedUserInfo by using a shared memory version counter")
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit 33d7292f29)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:08:00 +02:00
Thomas Lamprecht e9c2a34def REST: set error message extenesion for bad-request response log
We send it already to the user via the response body, but the
log_response does not has, nor wants to have FWIW, access to the
async body stream, so pass it through the ErrorMessageExtension
mechanism like we do else where.

Note that this is not only useful for PBS API proxy/daemon but also
the REST server of the file-restore daemon running inside the restore
VM, and it really is *very* helpful to debug things there..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit f4d371d2d2)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:07:41 +02:00
Thomas Lamprecht 0fad95f032 REST: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit 2d48533378)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:07:41 +02:00
Stoiko Ivanov 683595940b fix #3496: acme: plugin: add sleep for dns propagation
the dns plugin config allow for a specified amount of time to wait for
the TXT record to be set and propagated through DNS.

This patch adds a sleep for this amount of time.
The log message was taken from the perl implementation in proxmox-acme
for consistency.

Tested with the powerdns plugin in my test setup.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
(cherry picked from commit 3f84541412)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:06:29 +02:00
Stoiko Ivanov 40060c1fed config: acme: make validation_delay crate public
we need the setting in acme::plugin.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
(cherry picked from commit 4d8bd03668)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:06:29 +02:00
Stoiko Ivanov 2abee30fdd acme: plugin: fix error message
extract_challenge is used by both dns-01 and http-01 challenges.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
(cherry picked from commit f9bd5e1691)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:06:29 +02:00
Thomas Lamprecht 7cdc53bbf7 buildsys: docs: clean: also clean generated JS files
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit 13a2445744)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:06:04 +02:00
Fabian Ebner dac877252b api: disk list: sort by name
So callers get more stable results. Most noticeable, the disk list in
the web UI doesn't jump around upon reloading, and while sorting could
be done directly there, like this other callers get the benefit too.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
(cherry picked from commit bbff317aa7)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:04:57 +02:00
Fabian Ebner dd749b0e47 disks: also check for file systems with lsblk
Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
(cherry picked from commit 20429238e0)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:04:57 +02:00
Fabian Ebner f98c02cbc6 disks: refactor partition type handling
in preparation to also get the file system type from lsblk.

Co-developed-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
(cherry picked from commit 364299740f)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:04:57 +02:00
Thomas Lamprecht 218d7e3ec6 rest: log response: avoid unnecessary mut on variable
a match expresses the fallback slightly nicer and needs no mut,
which is always nice to avoid.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit 6b5013edb3)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:02:47 +02:00
Stefan Reiter acefa2bb6e auth: 'crypt' is not thread safe
According to crypt(3):
"crypt places its result in a static storage area, which will be
overwritten by subsequent calls to crypt. It is not safe to call crypt
from multiple threads simultaneously."

This means that multiple login calls as a PBS-realm user can collide and
produce intermittent authentication failures. A visible case is for
file-restore, where VMs with many disks lead to just as many auth-calls
at the same time, as the GUI tries to expand each tree element on load.

Instead, use the thread-safe variant 'crypt_r', which places the result
into a pre-allocated buffer of type 'crypt_data'. The C struct is laid
out according to 'lib/crypt.h.in' and the man page mentioned above.

Use the opportunity and make both arguments to the rust 'crypt' function
take a &[u8].

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
(cherry picked from commit c4c4b5a3ef)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 10:02:01 +02:00
Dietmar Maurer 36551172f3 depend on proxmox 0.11.6 (changed make_tmp_file() return type)
(cherry picked from commit bfd357c5a1)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 09:45:28 +02:00
Wolfgang Bumiller c26f4ef385 buildsys: Prepare new way for path dependencies
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
(cherry picked from commit 9f5b57a348)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 09:39:12 +02:00
Wolfgang Bumiller 60816a8a82 Cargo.toml: regroup imports
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
(cherry picked from commit aceae32baa)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 09:34:18 +02:00
Thomas Lamprecht d7d09712ef bump version to 1.1.12-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-09 12:58:14 +02:00
Thomas Lamprecht 825f019226 buildsys: call dpkg-buildpackage directly in deb-all
else we may double-build

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit a2c73c78dd)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-09 12:58:14 +02:00
Dominik Csapak ca5e5bb67f ui: datastore/OptionView: only navigate up when we removed the datastore
and not on window close

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
(cherry picked from commit 82cae19d19)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-09 12:54:50 +02:00
Dominik Csapak 8191ff150e ui: dashboard/DataStoreStatistics: fix closing <i> tag
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
(cherry picked from commit 4a489ae3de)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-09 12:49:42 +02:00
Thomas Lamprecht f2aeb13c68 subscription: set higher-level error to message instead of bailing
While the PVE one "bails" too, it has an eval around those and moves
the error to the message property, so lets do so too to ensure a user
can force an update on a too old subscription

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit b81818b6ad)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-09 12:48:03 +02:00
Dietmar Maurer ce76b4b3c2 bump version to 1.1-11-1 2021-06-30 11:25:11 +02:00
Dominik Csapak 44b9d6f162 tape/drive: fix logging when requesting media
we try to load the correct media in a loop until we find the correct tape.
when encountering an error or wrong tape, we want to log that (and send
an email if one is set) that requests the correct tape.

while trying to avoid printing the same errors more than once in a row,
we had at least one case (starting with an empty tape in the drive)
which would not print/send any tape request.

reworking that code to use a custom 'TapeRequest' enum, which contains
the state + error message, and a helper that prints and sends an email
when the state changes

this reduces the change check/log to a single variable, instead of 4
(tried, last_media_uuid, last_error, failure_reason)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-30 11:22:04 +02:00
Dietmar Maurer 53e80e8aa2 tape: fix LTO locate_file for HP drives
Add test code to the first locate_file command, compute locate_offset.
Subsequent locate_file commands use that offset.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-30 11:22:04 +02:00
Dominik Csapak f94aa5ceb1 fix #3393 (again): pxar/create: try to read xattrs/fcaps/acls by default
we have a static list of filesystems and their capabilities regarding
file attributes and fs features (e.g. sockets/fifos/etc) which also
includes xattrs,acls and fcaps

if we did not know a filesystem by its magic number (for example cephfs),
we did not even attempt to read xattrs, etc.

this patch adds those flags by default to unknown filesystems, and
removes them when we encounter EOPNOTSUPP (to remove the number
of syscalls)

with this, we should be able to catch xattrs/acls/fcaps on all
(unknown) fs types that support them

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-06-30 11:22:04 +02:00
Dominik Csapak 3e4b9868a0 proxmox-backup-manager: show task log on datastore create
since the output:
Result: "<UPID>"
is not really interesting, show instead the task log while
the datastore is creating, since it is now run in a worker

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-30 11:22:04 +02:00
Thomas Lamprecht 4d86df04a0 bump version to 1.1.10-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-16 09:55:47 +02:00
620 changed files with 36827 additions and 51465 deletions

View File

@ -1,6 +1,6 @@
[package] [package]
name = "proxmox-backup" name = "proxmox-backup"
version = "2.2.3" version = "1.1.14"
authors = [ authors = [
"Dietmar Maurer <dietmar@proxmox.com>", "Dietmar Maurer <dietmar@proxmox.com>",
"Dominik Csapak <d.csapak@proxmox.com>", "Dominik Csapak <d.csapak@proxmox.com>",
@ -15,61 +15,45 @@ edition = "2018"
license = "AGPL-3" license = "AGPL-3"
description = "Proxmox Backup" description = "Proxmox Backup"
homepage = "https://www.proxmox.com" homepage = "https://www.proxmox.com"
build = "build.rs"
exclude = [ "build", "debian", "tests/catar_data/test_symlink/symlink1"] exclude = [ "build", "debian", "tests/catar_data/test_symlink/symlink1"]
[workspace]
members = [
"pbs-buildcfg",
"pbs-client",
"pbs-config",
"pbs-datastore",
"pbs-fuse-loop",
"proxmox-rest-server",
"proxmox-rrd",
"pbs-tape",
"pbs-tools",
"proxmox-backup-banner",
"proxmox-backup-client",
"proxmox-file-restore",
"proxmox-restore-daemon",
"pxar-bin",
]
[lib] [lib]
name = "proxmox_backup" name = "proxmox_backup"
path = "src/lib.rs" path = "src/lib.rs"
[dependencies] [dependencies]
apt-pkg-native = "0.3.2" apt-pkg-native = "0.3.2"
base64 = "0.13" base64 = "0.12"
bitflags = "1.2.1" bitflags = "1.2.1"
bytes = "1.0" bytes = "1.0"
cidr = "0.2.1"
crc32fast = "1" crc32fast = "1"
endian_trait = { version = "0.6", features = ["arrays"] } endian_trait = { version = "0.6", features = ["arrays"] }
env_logger = "0.7"
flate2 = "1.0" flate2 = "1.0"
anyhow = "1.0" anyhow = "1.0"
foreign-types = "0.3"
thiserror = "1.0" thiserror = "1.0"
futures = "0.3" futures = "0.3"
h2 = { version = "0.3", features = [ "stream" ] } h2 = { version = "0.3", features = [ "stream" ] }
handlebars = "3.0" handlebars = "3.0"
hex = "0.4.3"
http = "0.2" http = "0.2"
hyper = { version = "0.14", features = [ "full" ] } hyper = { version = "0.14", features = [ "full" ] }
lazy_static = "1.4" lazy_static = "1.4"
libc = "0.2" libc = "0.2"
log = "0.4.17" log = "0.4"
nix = "0.24" nix = "0.19.1"
num-traits = "0.2" num-traits = "0.2"
once_cell = "1.3.1" once_cell = "1.3.1"
openssl = "0.10.38" # currently patched! openssl = "0.10"
pam = "0.7" pam = "0.7"
pam-sys = "0.5" pam-sys = "0.5"
percent-encoding = "2.1" percent-encoding = "2.1"
regex = "1.5.5" pin-utils = "0.1.0"
rustyline = "9" pin-project = "1.0"
regex = "1.2"
rustyline = "7"
serde = { version = "1.0", features = ["derive"] } serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0" serde_json = "1.0"
siphasher = "0.3" siphasher = "0.3"
@ -77,75 +61,32 @@ syslog = "4.0"
tokio = { version = "1.6", features = [ "fs", "io-util", "io-std", "macros", "net", "parking_lot", "process", "rt", "rt-multi-thread", "signal", "time" ] } tokio = { version = "1.6", features = [ "fs", "io-util", "io-std", "macros", "net", "parking_lot", "process", "rt", "rt-multi-thread", "signal", "time" ] }
tokio-openssl = "0.6.1" tokio-openssl = "0.6.1"
tokio-stream = "0.1.0" tokio-stream = "0.1.0"
tokio-util = { version = "0.7", features = [ "codec", "io" ] } tokio-util = { version = "0.6", features = [ "codec", "io" ] }
tower-service = "0.3.0" tower-service = "0.3.0"
udev = "0.4" udev = ">= 0.3, <0.5"
url = "2.1" url = "2.1"
#valgrind_request = { git = "https://github.com/edef1c/libvalgrind_request", version = "1.1.0", optional = true } #valgrind_request = { git = "https://github.com/edef1c/libvalgrind_request", version = "1.1.0", optional = true }
walkdir = "2" walkdir = "2"
webauthn-rs = "0.2.5"
xdg = "2.2" xdg = "2.2"
zstd = { version = "0.4", features = [ "bindgen" ] }
nom = "5.1" nom = "5.1"
crossbeam-channel = "0.5" crossbeam-channel = "0.5"
# Used only by examples currently:
zstd = { version = "0.6", features = [ "bindgen" ] }
pathpatterns = "0.1.2" pathpatterns = "0.1.2"
pxar = { version = "0.10.1", features = [ "tokio-io" ] } pxar = { version = "0.10.1", features = [ "tokio-io" ] }
proxmox-http = { version = "0.6.1", features = [ "client", "http-helpers", "websocket" ] } proxmox = { version = "0.11.6", features = [ "sortable-macro", "api-macro", "cli", "router", "tfa" ] }
proxmox-io = "1" proxmox-acme-rs = "0.3"
proxmox-lang = "1.1" proxmox-fuse = "0.1.1"
proxmox-router = { version = "1.2.2", features = [ "cli" ] } proxmox-http = { version = "0.2.1", features = [ "client", "http-helpers", "websocket" ] }
proxmox-schema = { version = "1.3.1", features = [ "api-macro" ] }
proxmox-section-config = "1"
proxmox-tfa = { version = "2", features = [ "api", "api-types" ] }
proxmox-time = "1.1.2"
proxmox-uuid = "1"
proxmox-serde = "0.1"
proxmox-shared-memory = "0.2"
proxmox-sys = { version = "0.3", features = [ "sortable-macro" ] }
proxmox-compression = "0.1"
proxmox-acme-rs = "0.4"
proxmox-apt = "0.8.0"
proxmox-async = "0.4"
proxmox-openid = "0.9.0"
pbs-api-types = { path = "pbs-api-types" }
pbs-buildcfg = { path = "pbs-buildcfg" }
pbs-client = { path = "pbs-client" }
pbs-config = { path = "pbs-config" }
pbs-datastore = { path = "pbs-datastore" }
proxmox-rest-server = { path = "proxmox-rest-server" }
proxmox-rrd = { path = "proxmox-rrd" }
pbs-tools = { path = "pbs-tools" }
pbs-tape = { path = "pbs-tape" }
# Local path overrides # Local path overrides
# NOTE: You must run `cargo update` after changing this for it to take effect! # NOTE: You must run `cargo update` after changing this for it to take effect!
[patch.crates-io] [patch.crates-io]
#proxmox-acme-rs = { path = "../proxmox-acme-rs" } #proxmox = { path = "../proxmox/proxmox", features = [ "sortable-macro", "api-macro", "cli", "router", "tfa" ] }
#proxmox-apt = { path = "../proxmox-apt" } #proxmox-http = { path = "../proxmox/proxmox-http", features = [ "client", "http-helpers", "websocket" ] }
#proxmox-async = { path = "../proxmox/proxmox-async" } #pxar = { path = "../pxar", features = [ "tokio-io" ] }
#proxmox-compression = { path = "../proxmox/proxmox-compression" }
#proxmox-borrow = { path = "../proxmox/proxmox-borrow" }
#proxmox-fuse = { path = "../proxmox-fuse" }
#proxmox-http = { path = "../proxmox/proxmox-http" }
#proxmox-io = { path = "../proxmox/proxmox-io" }
#proxmox-lang = { path = "../proxmox/proxmox-lang" }
#proxmox-openid = { path = "../proxmox-openid-rs" }
#proxmox-router = { path = "../proxmox/proxmox-router" }
#proxmox-schema = { path = "../proxmox/proxmox-schema" }
#proxmox-section-config = { path = "../proxmox/proxmox-section-config" }
#proxmox-shared-memory = { path = "../proxmox/proxmox-shared-memory" }
#proxmox-sys = { path = "../proxmox/proxmox-sys" }
#proxmox-serde = { path = "../proxmox/proxmox-serde" }
#proxmox-tfa = { path = "../proxmox/proxmox-tfa" }
#proxmox-time = { path = "../proxmox/proxmox-time" }
#proxmox-uuid = { path = "../proxmox/proxmox-uuid" }
#pxar = { path = "../pxar" }
[features] [features]
default = [] default = []

111
Makefile
View File

@ -17,8 +17,7 @@ USR_BIN := \
# Binaries usable by admins # Binaries usable by admins
USR_SBIN := \ USR_SBIN := \
proxmox-backup-manager \ proxmox-backup-manager
proxmox-backup-debug \
# Binaries for services: # Binaries for services:
SERVICE_BIN := \ SERVICE_BIN := \
@ -31,23 +30,6 @@ SERVICE_BIN := \
RESTORE_BIN := \ RESTORE_BIN := \
proxmox-restore-daemon proxmox-restore-daemon
SUBCRATES := \
pbs-api-types \
pbs-buildcfg \
pbs-client \
pbs-config \
pbs-datastore \
pbs-fuse-loop \
proxmox-rest-server \
proxmox-rrd \
pbs-tape \
pbs-tools \
proxmox-backup-banner \
proxmox-backup-client \
proxmox-file-restore \
proxmox-restore-daemon \
pxar-bin
ifeq ($(BUILD_MODE), release) ifeq ($(BUILD_MODE), release)
CARGO_BUILD_ARGS += --release CARGO_BUILD_ARGS += --release
COMPILEDIR := target/release COMPILEDIR := target/release
@ -75,15 +57,13 @@ RESTORE_DBG_DEB=proxmox-backup-file-restore-dbgsym_${DEB_VERSION}_${ARCH}.deb
DOC_DEB=${PACKAGE}-docs_${DEB_VERSION}_all.deb DOC_DEB=${PACKAGE}-docs_${DEB_VERSION}_all.deb
DEBS=${SERVER_DEB} ${SERVER_DBG_DEB} ${CLIENT_DEB} ${CLIENT_DBG_DEB} \ DEBS=${SERVER_DEB} ${SERVER_DBG_DEB} ${CLIENT_DEB} ${CLIENT_DBG_DEB} \
${RESTORE_DEB} ${RESTORE_DBG_DEB} ${DEBUG_DEB} ${DEBUG_DBG_DEB} ${RESTORE_DEB} ${RESTORE_DBG_DEB}
DSC = rust-${PACKAGE}_${DEB_VERSION}.dsc DSC = rust-${PACKAGE}_${DEB_VERSION}.dsc
DESTDIR= DESTDIR=
tests ?= --workspace all: cargo-build $(SUBDIRS)
all: $(SUBDIRS)
.PHONY: $(SUBDIRS) .PHONY: $(SUBDIRS)
$(SUBDIRS): $(SUBDIRS):
@ -95,23 +75,25 @@ test:
$(CARGO) test $(tests) $(CARGO_BUILD_ARGS) $(CARGO) test $(tests) $(CARGO_BUILD_ARGS)
doc: doc:
$(CARGO) doc --workspace --no-deps $(CARGO_BUILD_ARGS) $(CARGO) doc --no-deps $(CARGO_BUILD_ARGS)
# always re-create this dir # always re-create this dir
.PHONY: build .PHONY: build
build: build:
@echo "Setting pkg-buildcfg version to: $(DEB_VERSION_UPSTREAM)"
sed -i -e 's/^version =.*$$/version = "$(DEB_VERSION_UPSTREAM)"/' \
pbs-buildcfg/Cargo.toml
rm -rf build rm -rf build
mkdir build rm -f debian/control
cp -a debian \ debcargo package \
Cargo.toml src \ --config debian/debcargo.toml \
$(SUBCRATES) \ --changelog-ready \
docs etc examples tests www zsh-completions \ --no-overlay-write-back \
defines.mk Makefile \ --directory build \
./build/ proxmox-backup \
rm -f build/Cargo.lock $(shell dpkg-parsechangelog -l debian/changelog -SVersion | sed -e 's/-.*//')
sed -e '1,/^$$/ ! d' build/debian/control > build/debian/control.src
cat build/debian/control.src build/debian/control.in > build/debian/control
rm build/debian/control.in build/debian/control.src
cp build/debian/control debian/control
rm build/Cargo.lock
find build/debian -name "*.hint" -delete find build/debian -name "*.hint" -delete
$(foreach i,$(SUBDIRS), \ $(foreach i,$(SUBDIRS), \
$(MAKE) -C build/$(i) clean ;) $(MAKE) -C build/$(i) clean ;)
@ -141,61 +123,27 @@ $(DSC): build
cd build; dpkg-buildpackage -S -us -uc -d -nc cd build; dpkg-buildpackage -S -us -uc -d -nc
lintian $(DSC) lintian $(DSC)
.PHONY: clean distclean deb clean
distclean: clean distclean: clean
clean: clean-deb
clean:
$(foreach i,$(SUBDIRS), \ $(foreach i,$(SUBDIRS), \
$(MAKE) -C $(i) clean ;) $(MAKE) -C $(i) clean ;)
$(CARGO) clean $(CARGO) clean
rm -f .do-cargo-build rm -rf *.deb *.dsc *.tar.gz *.buildinfo *.changes build
find . -name '*~' -exec rm {} ';' find . -name '*~' -exec rm {} ';'
# allows one to avoid running cargo clean when one just wants to tidy up after a packgae build
clean-deb:
rm -rf *.deb *.dsc *.tar.gz *.buildinfo *.changes build/
.PHONY: dinstall .PHONY: dinstall
dinstall: ${SERVER_DEB} ${SERVER_DBG_DEB} ${CLIENT_DEB} ${CLIENT_DBG_DEB} \ dinstall: ${SERVER_DEB} ${SERVER_DBG_DEB} ${CLIENT_DEB} ${CLIENT_DBG_DEB}
${DEBUG_DEB} ${DEBUG_DBG_DEB}
dpkg -i $^ dpkg -i $^
# make sure we build binaries before docs # make sure we build binaries before docs
docs: $(COMPILEDIR)/dump-catalog-shell-cli $(COMPILEDIR)/docgen docs: cargo-build
.PHONY: cargo-build .PHONY: cargo-build
cargo-build: cargo-build:
rm -f .do-cargo-build $(CARGO) build $(CARGO_BUILD_ARGS)
$(MAKE) $(COMPILED_BINS)
$(COMPILED_BINS) $(COMPILEDIR)/dump-catalog-shell-cli $(COMPILEDIR)/docgen: .do-cargo-build
.do-cargo-build:
$(CARGO) build $(CARGO_BUILD_ARGS) \
--package proxmox-backup-banner \
--bin proxmox-backup-banner \
--package proxmox-backup-client \
--bin proxmox-backup-client \
--bin dump-catalog-shell-cli \
--bin proxmox-backup-debug \
--package proxmox-file-restore \
--bin proxmox-file-restore \
--package pxar-bin \
--bin pxar \
--package pbs-tape \
--bin pmt \
--bin pmtx \
--package proxmox-restore-daemon \
--bin proxmox-restore-daemon \
--package proxmox-backup \
--bin docgen \
--bin proxmox-backup-api \
--bin proxmox-backup-manager \
--bin proxmox-backup-proxy \
--bin proxmox-daily-update \
--bin proxmox-file-restore \
--bin proxmox-tape \
--bin sg-tape-cmd
touch "$@"
$(COMPILED_BINS): cargo-build
.PHONY: lint .PHONY: lint
lint: lint:
@ -223,11 +171,10 @@ install: $(COMPILED_BINS)
$(MAKE) -C docs install $(MAKE) -C docs install
.PHONY: upload .PHONY: upload
upload: ${SERVER_DEB} ${CLIENT_DEB} ${RESTORE_DEB} ${DOC_DEB} ${DEBUG_DEB} upload: ${SERVER_DEB} ${CLIENT_DEB} ${RESTORE_DEB} ${DOC_DEB}
# check if working directory is clean # check if working directory is clean
git diff --exit-code --stat && git diff --exit-code --stat --staged git diff --exit-code --stat && git diff --exit-code --stat --staged
tar cf - ${SERVER_DEB} ${SERVER_DBG_DEB} ${DOC_DEB} ${CLIENT_DEB} \ tar cf - ${SERVER_DEB} ${SERVER_DBG_DEB} ${DOC_DEB} ${CLIENT_DEB} ${CLIENT_DBG_DEB} | \
${CLIENT_DBG_DEB} ${DEBUG_DEB} ${DEBUG_DBG_DEB} \ ssh -X repoman@repo.proxmox.com upload --product pbs --dist buster
| ssh -X repoman@repo.proxmox.com upload --product pbs --dist bullseye tar cf - ${CLIENT_DEB} ${CLIENT_DBG_DEB} | ssh -X repoman@repo.proxmox.com upload --product "pve,pmg,pbs-client" --dist buster
tar cf - ${CLIENT_DEB} ${CLIENT_DBG_DEB} | ssh -X repoman@repo.proxmox.com upload --product "pve,pmg,pbs-client" --dist bullseye tar cf - ${RESTORE_DEB} ${RESTORE_DBG_DEB} | ssh -X repoman@repo.proxmox.com upload --product "pve" --dist buster
tar cf - ${RESTORE_DEB} ${RESTORE_DBG_DEB} | ssh -X repoman@repo.proxmox.com upload --product "pve" --dist bullseye

View File

@ -1,7 +1,3 @@
Build & Release Notes
*********************
``rustup`` Toolchain ``rustup`` Toolchain
==================== ====================
@ -44,44 +40,41 @@ example for proxmox crate above).
Build Build
===== =====
on Debian 11 Bullseye on Debian Buster
Setup: Setup:
1. # echo 'deb http://download.proxmox.com/debian/devel/ bullseye main' | sudo tee /etc/apt/sources.list.d/proxmox-devel.list 1. # echo 'deb http://download.proxmox.com/debian/devel/ buster main' >> /etc/apt/sources.list.d/proxmox-devel.list
2. # sudo wget https://enterprise.proxmox.com/debian/proxmox-release-bullseye.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg 2. # sudo wget http://download.proxmox.com/debian/proxmox-ve-release-6.x.gpg -O /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg
3. # sudo apt update 3. # sudo apt update
4. # sudo apt install devscripts debcargo clang 4. # sudo apt install devscripts debcargo clang
5. # git clone git://git.proxmox.com/git/proxmox-backup.git 5. # git clone git://git.proxmox.com/git/proxmox-backup.git
6. # cd proxmox-backup; sudo mk-build-deps -ir 6. # sudo mk-build-deps -ir
Note: 2. may be skipped if you already added the PVE or PBS package repository Note: 2. may be skipped if you already added the PVE or PBS package repository
You are now able to build using the Makefile or cargo itself, e.g.:: You are now able to build using the Makefile or cargo itself.
# make deb-all
# # or for a non-package build
# cargo build --all --release
Design Notes Design Notes
************ ============
Here are some random thought about the software design (unless I find a better place). Here are some random thought about the software design (unless I find a better place).
Large chunk sizes Large chunk sizes
================= -----------------
It is important to notice that large chunk sizes are crucial for performance. It is important to notice that large chunk sizes are crucial for
We have a multi-user system, where different people can do different operations performance. We have a multi-user system, where different people can do
on a datastore at the same time, and most operation involves reading a series different operations on a datastore at the same time, and most operation
of chunks. involves reading a series of chunks.
So what is the maximal theoretical speed we can get when reading a series of So what is the maximal theoretical speed we can get when reading a
chunks? Reading a chunk sequence need the following steps: series of chunks? Reading a chunk sequence need the following steps:
- seek to the first chunk's start location - seek to the first chunk start location
- read the chunk data - read the chunk data
- seek to the next chunk's start location - seek to the first chunk start location
- read the chunk data - read the chunk data
- ... - ...

23
build.rs Normal file
View File

@ -0,0 +1,23 @@
// build.rs
use std::env;
use std::process::Command;
fn git_command(args: &[&str]) -> String {
match Command::new("git").args(args).output() {
Ok(output) => String::from_utf8(output.stdout).unwrap().trim_end().to_string(),
Err(err) => {
panic!("git {:?} failed: {}", args, err);
}
}
}
fn main() {
let repo_path = git_command(&["rev-parse", "--show-toplevel"]);
let repoid = match env::var("REPOID") {
Ok(repoid) => repoid,
Err(_) => git_command(&["rev-parse", "HEAD"]),
};
println!("cargo:rustc-env=REPOID={}", repoid);
println!("cargo:rerun-if-changed={}/.git/HEAD", repo_path);
}

774
debian/changelog vendored
View File

@ -1,734 +1,39 @@
rust-proxmox-backup (2.2.3-1) bullseye; urgency=medium rust-proxmox-backup (1.1.14-1) buster; urgency=medium
* datastore: swap dirtying the datastore cache every 60s by just using the * drop RawWaker usage to avoid a leaking a refcount
available config digest to detect any changes accuratly when the actually
happen
* api: datastore list and datastore status: avoid opening datastore and * pbs-tools: LruCache: implement Drop to fix a memory leak for the cache
possibly iterating over namespace (for lesser privileged users), but
rather use the in-memory ACL tree directly to check if there's access to
any namespace below.
-- Proxmox Support Team <support@proxmox.com> Sat, 04 Jun 2022 16:30:05 +0200 * ui: add notice for nearing PBS 1.1 End-of-Life
rust-proxmox-backup (2.2.2-3) bullseye; urgency=medium * backport "datastore: lookup: reuse ChunkStore on stale datastore re-open"
* datastore: lookup: reuse ChunkStore on stale datastore re-open -- Proxmox Support Team <support@proxmox.com> Thu, 02 Jun 2022 18:07:54 +0200
* bump tokio (async framework) dependency rust-proxmox-backup (1.1.13-3) buster; urgency=medium
-- Proxmox Support Team <support@proxmox.com> Thu, 02 Jun 2022 17:25:01 +0200 * fix sending log-rotation command to API daemons
rust-proxmox-backup (2.2.2-2) bullseye; urgency=medium -- Proxmox Support Team <support@proxmox.com> Tue, 19 Oct 2021 10:21:18 +0200
* improvement of error handling when removing status files and locks from rust-proxmox-backup (1.1.13-2) buster; urgency=medium
jobs that were never executed.
-- Proxmox Support Team <support@proxmox.com> Wed, 01 Jun 2022 16:22:22 +0200 * revert "auth: improve thread safety of 'crypt' C-library", not safe for
Debian buster based releases.
rust-proxmox-backup (2.2.2-1) bullseye; urgency=medium -- Proxmox Support Team <support@proxmox.com> Mon, 26 Jul 2021 16:40:07 +0200
* Revert "verify: allow '0' days for reverification", was already possible rust-proxmox-backup (1.1.13-1) buster; urgency=medium
by setting "ignore-verified" to false
* ui: datastore permissions: allow ACL path edit & query namespaces
* accessible group iter: allow NS descending with DATASTORE_READ privilege
* prune datastore: rework worker tak log
* prune datastore: support max-depth and improve priv checks
* ui: prune input: support opt-in recursive/max-depth field
* add prune job config and api, allowing one to setup a scheduled pruning
for a specific namespace only
* ui: add ui for prune jobs
* api: disable setting prune options in datastore.cfg and transform any
existing prune tasks from datastore config to new prune job config in a
post installation hook
* proxmox-tape: use correct api call for 'load-media-from-slot'
* avoid overly strict privilege restrictions for some API endpoints and
actions when using namespaces. Better support navigating the user
interface when only having Datastore.Admin on a (sub) namespace.
* include required privilege names in some permission errors
* docs: fix some typos
* api: status: include empty entry for stores with ns-only privs
* ui: datastore options: avoid breakage if rrd store ore active-ops cannot
be queried
* ui: datastore content: only mask the inner treeview, not the top bar on
error to allow a user to trigger a manual reload
* ui: system config: improve bottom margins and scroll behavior
-- Proxmox Support Team <support@proxmox.com> Wed, 01 Jun 2022 15:09:36 +0200
rust-proxmox-backup (2.2.1-1) bullseye; urgency=medium
* docs: update some screenshots and add new ones
* docs: port overcertificate management chapters from Proxmox VE
* ui: datastore/Summary: correctly show the io-delay chart
* ui: sync/verify jobs: use pmxDisplayEditField to fix editing
* ui: server status: use power of two base for memory and swap
* ui: use base 10 (SI) for all storage related displays
* ui: datastore selector: show maintenance mode in selector
* docs: basic maintenance mode section
* docs: storage: refer to options
* storage: add some initial namespace docs
* ui: tape restore: fix form validation for datastore mapping
* ui: namespace selector: show picker empty text if no namespace
-- Proxmox Support Team <support@proxmox.com> Tue, 17 May 2022 13:56:50 +0200
rust-proxmox-backup (2.2.0-2) bullseye; urgency=medium
* client: add CLI auto-completion callbacks for ns parameters
* ui: fix setting protection in namespace
* ui: switch summary repo status to widget toolkit one
* ui: verify outdated: disallow blank and drop wrong empty text
* docs: add namespace section to sync documentation
* ui: datastore summary: add maintenance mask for offline entries
* ui: verify/sync: allow to optionally override ID again
* prune: fix workerid issues
-- Proxmox Support Team <support@proxmox.com> Mon, 16 May 2022 19:01:13 +0200
rust-proxmox-backup (2.2.0-1) bullseye; urgency=medium
* cli: improve namespace integration in proxmox-backup-client and
proxmox-tape
* docs: tape: add information about namespaces
* api: datastore status: make counts for groups and snapshots iterate over
all accessible namespaces recursively
* ui: fix storeId casing to register store correctly, so that we can query
it again for the ACL permission path selector
* ui: trigger datastore update after maintenance mode edit
* ui: namespace selector: set queryMode to local to avoid bogus background
requests on typing
* ui: sync job: fix clearing value of remote target-namespace by mistake on
edit
* ui: remote target ns selector: add clear trigger
* ui: prune group: add namespace info to title
* fix #4001: ui: add prefix to files downloaded through the pxar browser
* ui: datastore: reload content tree on successful datastore add
* ui: datastore: allow deleting currently shown namespace
* docs: rework access control, list available privileges
* docs: access control: add "Objects and Paths" section and fix
add-permission screenshot
-- Proxmox Support Team <support@proxmox.com> Mon, 16 May 2022 11:06:05 +0200
rust-proxmox-backup (2.1.10-1) bullseye; urgency=medium
* datastore: drop bogus chunk size check, can cause trouble
* pull/sync: detect remote lack of namespace support
* pull/sync: correctly query with remote-ns as parent
* ui: sync: add reduced max-depth selector
* ui: group filter: make also local filter NS aware
* api types: set NS_MAX_DEPTH schema default to MAX_NAMESPACE_DEPTH instead
of 0
* tape: notify when arriving at end of media
* tree-wide: rename 'backup-ns' API parameters to 'ns'
* tape: add namespaces/recursion depth to tape backup jobs
* api: tape/restore: add namespace mapping
* tape: bump catalog/snapshot archive magic
* ui: tape: backup overview: show namespaces as their own level above groups
* ui: tape restore: allow simple namespace mapping
-- Proxmox Support Team <support@proxmox.com> Fri, 13 May 2022 14:26:32 +0200
rust-proxmox-backup (2.1.9-2) bullseye; urgency=medium
* api: tape restore: lock the target datastore, not the source one
* chunk store: force write chunk again if it exist but its metadata length
is zero
* completion: fix 'group-filter' parameter name
* implement backup namespaces for datastores, allowing one to reuse a single
chunkstore deduplication domain for multiple sources without naming
conflicts and with fine-grained access control.
* make various datastore related API calls backup namespace aware
* make sync and pull backup namespace aware
* ui: datastore content: show namespaces but only one level at a time
* ui: make various datastore related UI components namespace aware
* fix various bugs, add namespace support to file-restore
-- Proxmox Support Team <support@proxmox.com> Thu, 12 May 2022 14:25:53 +0200
rust-proxmox-backup (2.1.8-1) bullseye; urgency=medium
* api: status: return gc-status again
* proxmox-backup-proxy: stop accept() loop on daemon shutdown to avoid that
new request get accepted while the REST stack is already stopped, for
example on the reload triggered by a package upgrade.
* pull: improve filtering local removal candidates
-- Proxmox Support Team <support@proxmox.com> Mon, 02 May 2022 17:36:11 +0200
rust-proxmox-backup (2.1.7-1) bullseye; urgency=medium
* pbs-tape: sgutils2: check sense data when status is 'CHECK_CONDITION'
* rework & refactor datastore implementation for a more hierarchical access
structure
* datastore: implement Iterator for backup group and snapshot listing to
allow more efficient access for cases where we do not need the whole list
in memory
* pbs-client: extract: rewrite create_zip with sequential decoder
* pbs-client: extract: add top-level dir in tar.zst
* fix #3067: ui: add a separate notes view for longer markdown notes and
copy the markdown primer from Proxmox VE to Proxmox Backup Server docs
* restore-daemon: start disk initialization in parallel to the api
* restore-daemon: put blocking code into 'block_in_place'
* restore-daemon: avoid auto-pre-mounting zpools completely, the upfront
(time) cost can be to big to pay up initially, e.g., if there are many
subvolumes present, so only mount on demand.
* file-restore: add 'timeout' and 'json-error' parameter
* ui: add summary mask when in maintenance mode
* ui: update datastore's navigation icon and tooltip if it is in maintenance
mode
-- Proxmox Support Team <support@proxmox.com> Wed, 27 Apr 2022 19:53:53 +0200
rust-proxmox-backup (2.1.6-1) bullseye; urgency=medium
* api: verify: allow passing '0 days' for immediate re-verification
* fix #3103. node configuration: allow to configure default UI language
* fix #3856: tape: encryption key's password hint parameter is not optional
* re-use PROXMOX_DEBUG environment variable to control log level filter
* ui: WebAuthn: fix stopping store upgrades on destroy and decrease interval
* report: add tape, traffic control and disk infos and tune output order
* fix #3853: cli/api: add force option to tape key change-passphrase
* fix #3323: cli client: add dry-run option for backup command
* tape: make iterating over chunks to backup smarter to avoid some work
* bin: daily-update: make single checks/updates fail gracefully and log
to syslog directly instead of going through stdout indirectly.
* datastore: allow to turn of inode-sorting for chunk iteration. While inode
sorting benefits read-performance on block devices with higher latency
(e.g., spinning disks), it's also some extra work to get the metadata
required for sorting, so its a trade-off. For setups that have either very
slow or very fast metadata IO the benefits may turn into a net cost.
* docs: explain retention time for event allocation policy in more detail
* docs: add tape schedule examples
* proxmox-backup-debug api: parse parameters before sending to api
* ui: fix panel height in the dashboard for three-column view mode
* fix #3934 tape owner-selector to auth-id (user OR token)
* fix #3067: api: add support for multi-line comments in the node
configuration
* pbs-client: print error when we couldn't download previous FIDX/DIDX for
incremental change tracking
* fix #3854 add command to import a key from a file (json or paper-key
format) to proxmox-tape
* improve IO access pattern for some scenarios like TFA with high user and
login count or the file-restore-for-block-backup VM's internal driver.
* pxar create: fix anchored path pattern matching when adding entries
* docs: client: file exclusion: add note about leading slash
* rest-server: add option to rotate task logs by 'max_days' instead of
'max_files'
* pbs-datastore: add active operations tracking and use it to implement a
graceful transition into the also newly added maintenance mode (read-only
or offline) for datastores. Note that the UI implementation may still show
some rough edges if a datastore is in offline mode for maintenance.
* add new streaming-response type for API call responses and enable it for
the snapshot and task-log list, which can both get rather big. This avoids
allocation of a potentially big intermediate memory buffer and thus
overall memory usage.
* pxar: accompany existing .zip download support with a tar.zst(d) one. The
tar archive supports more file types (e.g., hard links or device nodes)
and zstd allows for a efficient but still effective compression.
-- Proxmox Support Team <support@proxmox.com> Wed, 13 Apr 2022 17:00:53 +0200
rust-proxmox-backup (2.1.5-1) bullseye; urgency=medium
* tell system allocator to always use mmap for allocations >= 128 KiB to
improve reclaimability of free'd memory to the OS and reduce peak and avg.
RSS consumption
* file restore: always wait up to 25s for the file-restore-VM to have
scanned all possible filesystems in a backup. While theoretically there
are some edge cases where the tool waits less now, most common ones should
be waiting more compared to the 12s "worst" case previously.
-- Proxmox Support Team <support@proxmox.com> Wed, 26 Jan 2022 16:23:09 +0100
rust-proxmox-backup (2.1.4-1) bullseye; urgency=medium
* config: add tls ciphers to NodeConfig
* pbs-tools: improve memory foot print of LRU Cache
* update dependencies to avoid a ref-count leak in async helpers
-- Proxmox Support Team <support@proxmox.com> Fri, 21 Jan 2022 10:48:14 +0100
rust-proxmox-backup (2.1.3-1) bullseye; urgency=medium
* fix #3618: proxmox-async: zip: add conditional EFS flag to zip files to
improve non-ascii code point extraction under windows.
* OpenID Connect login: improve error message for disabled users
* ui: tape: backup job: add second tab for group-filters to add/edit window
* ui: sync job: add second tab for group-filters to add/edit window
* ui: calendar event: add once daily example and clarify the workday one
* fix #3794: api types: set backup time (since the UNIX epoch) lower limit
to 1
* ui: fix opening settings window in datastore panel
* api: zfs: create zpool with `relatime=on` flag set
* fix #3763: disable SSL/TLS renegotiation
* node config: add email-from parameter to control notification sender
address
* ui: configuration: rename the "Authentication" tab to "Other" and add a
"General" section with HTTP-proxy and email-from settings
* datastore stats: not include the unavailable `io_ticks` for ZFS
datastores
* ui: hide RRD chart for IO delay if no `io_ticks` are returned
* fix #3058: ui: improve remote edit UX by clarifying ID vs host fields
* docs: fix some minor typos
* api-types: relax nodename API schema, make it a simple regex check like in
Proxmox VE
-- Proxmox Support Team <support@proxmox.com> Wed, 12 Jan 2022 16:49:13 +0100
rust-proxmox-backup (2.1.2-1) bullseye; urgency=medium
* docs: backup-client: fix wrong reference
* docs: remotes: note that protected flags will not be synced
* sync job: correctly apply rate limit
-- Proxmox Support Team <support@proxmox.com> Tue, 23 Nov 2021 13:56:15 +0100
rust-proxmox-backup (2.1.1-2) bullseye; urgency=medium
* docs: update and add traffic control related screenshots
* docs: mention traffic control (bandwidth limits) for sync jobs
-- Proxmox Support Team <support@proxmox.com> Mon, 22 Nov 2021 16:07:39 +0100
rust-proxmox-backup (2.1.1-1) bullseye; urgency=medium
* fix proxmox-backup-manager sync-job list
* ui, api: sync-job: allow one to configure a rate limit
* api: snapshot list: set default for 'protected' flag
* ui: datastore content: rework rendering protection state
* docs: update traffic control docs (use HumanBytes)
* ui: traffic-control: include ipv6 in 'all' networks
* ui: traffic-control edit: add spaces between networks for more
readabillity
* tape: fix passing-through key-fingerprint
* avoid a bogus error regarding logrotate-path due to a reversed check
-- Proxmox Support Team <support@proxmox.com> Mon, 22 Nov 2021 12:24:31 +0100
rust-proxmox-backup (2.1.0-1) bullseye; urgency=medium
* rest server: make successful-ticket auth log a debug one to avoid
syslog spam
* traffic-controls: add API/CLI to show current traffic
* docs: add traffic control section
* ui: use TFA widgets from widget toolkit
* sync: allow pulling groups selectively
* fix #3533: tape backup: filter groups according to config
* proxmox-tape: add missing notify-user option to backup command
* openid: allow arbitrary username-claims
* openid: support configuring the prompt, scopes and ACR values
* use human-byte for traffic-control rate-in/out and burst-in/out config
* ui: add traffic control view and editor
-- Proxmox Support Team <support@proxmox.com> Sat, 20 Nov 2021 22:44:07 +0100
rust-proxmox-backup (2.0.14-1) bullseye; urgency=medium
* fix directory permission problems
* add traffic control configuration config with API
* proxmox-backup-proxy: implement traffic control
* proxmox-backup-client: add rate/burst parameter to backup/restore CLI
* openid_login: vertify that firstname, lastname and email fits our
schema definitions
* docs: add info about protection flag to client docs
* fix #3602: ui: datastore/Content: add action to set protection status
* ui: add protected icon to snapshot (if they are protected)
* ui: PruneInputPanel: add keepReason 'protected' for protected backups
* proxmox-backup-client: add 'protected' commands
* acme: interpret no TOS as accepted
* acme: new_account: prevent replacing existing accounts
-- Proxmox Support Team <support@proxmox.com> Fri, 12 Nov 2021 08:04:55 +0100
rust-proxmox-backup (2.0.13-1) bullseye; urgency=medium
* tape: simplify export_media_set for pool writer
* tape: improve export_media error message for not found tape
* rest-server: use hashmap for parameter errors
* proxmox-rrd: use new file firmat with higher resolution
* proxmox-rrd: use a journal to reduce amount of bytes written
* use new fsync parameter to replace_file and atomic_open_or_create
* docs: langauge and formatting fixup
* docs: Update for new features/functionality
-- Proxmox Support Team <support@proxmox.com> Thu, 21 Oct 2021 08:17:00 +0200
rust-proxmox-backup (2.0.12-1) bullseye; urgency=medium
* proxmox-backup-proxy: clean up old tasks when their reference was rotated
out of the task-log index
* api daemons: fix sending log-reopen command
-- Proxmox Support Team <support@proxmox.com> Tue, 19 Oct 2021 10:48:28 +0200
rust-proxmox-backup (2.0.11-1) bullseye; urgency=medium
* drop aritifical limits for task-UPID length
* tools: smart: only throw error for the fatal usage errors of smartctl
* api: improve returning errors for extjs formatter
* proxmox-rest-server: improve logging
* subscription: switch verification domain over to shop.proxmox.com
* rest-server/daemon: use new sd_notify_barrier helper for handling
synchronization with systemd on service reloading
* ui: datastore/Content: add empty text for no snapshots
* ui: datastore/Content: move first store-load into activate listener to
ensure we've a proper loading mask for better UX
-- Proxmox Support Team <support@proxmox.com> Tue, 05 Oct 2021 16:34:14 +0200
rust-proxmox-backup (2.0.10-1) bullseye; urgency=medium
* ui: fix order of prune keep reasons
* server: add proxmox-backup-debug binary with chunk/file inspection, an API
shell with completion support
* restructured code base to reduce linkage and libraray ABI version
constraints for all non-server binaries (client, pxar, file-restore)
* zsh: fix passign parameters in auto-completion scripts
* tape: also add 'force-media-set' to availablea CLI options
* api: nodes: add missing node list (index) api endpoint
* docs: proxmox-backup-debug: add info about the new 'api' subcommand
* docs/technical-overview: add troubleshooting section
-- Proxmox Support Team <support@proxmox.com> Tue, 21 Sep 2021 14:00:48 +0200
rust-proxmox-backup (2.0.9-2) bullseye; urgency=medium
* tape backup: mention groups that were empty
* tape: compute next-media-label for each tape backup job
* tape: lto: increase default timeout to 10 minutes
* ui: display next-media-label for tape backup jobs
* cli: proxmox-tape backup-job list: use status api and display next-run
and next-media-label
-- Proxmox Support Team <support@proxmox.com> Tue, 24 Aug 2021 14:44:12 +0200
rust-proxmox-backup (2.0.8-1) bullseye; urgency=medium
* use proxmox-apt to 0.6
* api: apt: adapt to proxmox-apt back-end changes
* api/ui: allow zstd compression for new zpools
* tape: media_catalog: add snapshot list cache for catalog
* api2: tape: media: use MediaCatalog::snapshot_list for content listing
* tape: lock media_catalog file to to get a consistent view with load_catalog
* tape: changer: handle libraries that sends wrong amount of data
* tape: changer: remove unnecesary inquiry parameter
* api2: tape/restore: commit temporary catalog at the end
* docs: tape: add instructions on how to restore the catalog
* ui: tape/ChangerStatus: improve layout for large libraries
* tape: changer: handle invalid descriptor data from library in status page
* datastore config: cleanup code (use flatten attribute)
-- Proxmox Support Team <support@proxmox.com> Mon, 02 Aug 2021 10:34:55 +0200
rust-proxmox-backup (2.0.7-1) bullseye; urgency=medium
* tape changer: better cope with models that are not following spec
proposals when returning the status page
* tape changer: make DVCID information optional, not all devices return it
* restore daemon: setup the 'backup' system user and group in the minimal
restore environment, as we like to ensure that all state files are ownend
by them.
-- Proxmox Support Team <support@proxmox.com> Fri, 23 Jul 2021 08:43:51 +0200
rust-proxmox-backup (2.0.6-1) bullseye; urgency=medium
* increase maximum drives per changer to 255
* allow one to pass a secret not only directly through the environment value,
but also indirectly through a file path, an open file descriptor or a
command that can write the secret to standard out.
* pull in new proxmox library version to improve the file system
comaptibility on creation of atomic files, e.g., lock files.
-- Proxmox Support Team <support@proxmox.com> Thu, 22 Jul 2021 10:22:19 +0200
rust-proxmox-backup (2.0.5-2) bullseye; urgency=medium
* ui: tape: backup overview: increase timeout for media-set content
* tape: changer: always retry until timeout
* file-restore: increase lock timeout on QEMU map
* fix #3515: file-restore-daemon: allow LVs/PVs with dash in name
* fix #3526: correctly filter tasks with 'since' and 'until'
* tape: changer: make scsi request for DVCID a separate one, as some
libraries cannot handle requesting that combined with volume tags in one
go
* api, ui: datastore: add new 'prune-datastore' api call and expose it with
a 'Prune All' button
* make creating log files more robust so that theys are always owned by the
less privileged `backup` user
-- Proxmox Support Team <support@proxmox.com> Wed, 21 Jul 2021 09:12:39 +0200
rust-proxmox-backup (2.0.4-1) bullseye; urgency=medium
* change tape drive lock path to avoid issues with sticky bit on tmpfs
mountpoint
* tape: changer: query transport-element types separately
* auth: improve thread safety of 'crypt' C-library * auth: improve thread safety of 'crypt' C-library
-- Proxmox Support Team <support@proxmox.com> Mon, 12 Jul 2021 18:51:21 +0200 * file-restore: increase lock timeout on QEMU map
rust-proxmox-backup (2.0.3-1) bullseye; urgency=medium
* api: apt: add repositories info and update calls
* ui: administration: add APT repositories status and update panel
* api: access domains: add get/create/update/delete endpoints for realms
* ui: access control: add 'Realm' tab for adding and editing OpenID Connect
identity provider
* fix #3447: ui: Dashboard: disallow selection of datastore statistics row
* ui: tapeRestore: make window non-resizable
* ui: dashboard: rework resource-load panel to a more detailed status panel,
showing, among other things, uptime, Kernel version, CPU info and
repository status.
* ui: adminsitration/dashboard: auto-scale columns count and add
browser-local setting to override that to a fixed value of columns.
* fix #3212: api, ui: add support for notes on backup groups
-- Proxmox Support Team <support@proxmox.com> Mon, 12 Jul 2021 08:07:41 +0200
rust-proxmox-backup (2.0.2-1) bullseye; urgency=medium
* ui: use task list component from widget toolkit
* api: add keep-job-configs flag to datastore remove endpoint
* api: config: delete datastore: also remove tape backup jobs
* ui: tape restore: mark datastore selector as 'not a form field' to fix
compatibility with ExtJS 7.0
* ui: datastore removal: only navigate away when the user actually confirmed
the removal of that datastore
-- Proxmox Support Team <support@proxmox.com> Thu, 08 Jul 2021 14:44:12 +0200
rust-proxmox-backup (2.0.1-2) bullseye; urgency=medium
* file restore daemon: log basic startup steps * file restore daemon: log basic startup steps
* REST-API: set error message extension for bad-request response log to * REST-API: set error message extension for bad-request response log to
ensure the actual error is logged in any (access) log, making debugging ensure the actual error is logged in any (access) log, making debugging
such issues easier such issues easier.
* restore daemon: create /run/proxmox-backup on startup as there's now some
runtime state saved there, which failed all API requests to the restore
daemon otherwise
* restore daemon: use millisecond log resolution * restore daemon: use millisecond log resolution
@ -736,19 +41,33 @@ rust-proxmox-backup (2.0.1-2) bullseye; urgency=medium
ensuring DNS propagation of that record. This makes it catch up with the ensuring DNS propagation of that record. This makes it catch up with the
docs/web-interface, where the option was already available. docs/web-interface, where the option was already available.
* docs: initial update to repositories for bullseye -- Proxmox Support Team <support@proxmox.com> Fri, 23 Jul 2021 12:34:29 +0200
-- Proxmox Support Team <support@proxmox.com> Sat, 03 Jul 2021 23:14:49 +0200 rust-proxmox-backup (1.1.12-1) buster; urgency=medium
rust-proxmox-backup (2.0.0-2) bullseye; urgency=medium * subscription: set higher-level error to message instead of bailing out, to
ensure a force-check gets through
* file-restore-daemon/disk: add LVM (thin) support * ui: dashboard: datastore stats: fix closing <i> tag
-- Proxmox Support Team <support@proxmox.com> Sat, 03 Jul 2021 02:15:16 +0200 * ui: datastore: option view: only navigate up when we actually removed the
datastore
rust-proxmox-backup (2.0.0-1) bullseye; urgency=medium -- Proxmox Support Team <support@proxmox.com> Fri, 09 Jul 2021 12:56:35 +0200
* initial bump for Debian 11 Bullseye / Proxmox Backup Server 2.0 rust-proxmox-backup (1.1.11-1) buster; urgency=medium
* tape/drive: fix logging when requesting media
* tape: fix LTO locate_file for HP drives
* fix #3393 (again): pxar/create: try to read xattrs/fcaps/acls by default
* proxmox-backup-manager: show task log on datastore create
-- Proxmox Support Team <support@proxmox.com> Wed, 30 Jun 2021 11:24:20 +0200
rust-proxmox-backup (1.1.10-1) buster; urgency=medium
* ui: datastore list summary: catch and show errors per datastore * ui: datastore list summary: catch and show errors per datastore
@ -765,7 +84,7 @@ rust-proxmox-backup (2.0.0-1) bullseye; urgency=medium
* ui: datastore options: add remove button to drop a datastore from the * ui: datastore options: add remove button to drop a datastore from the
configuration, without removing any actual data configuration, without removing any actual data
* ui: tape: drive selector: do not auto select the drive * ui: tape: drive selector: do not autoselect the drive
* ui: tape: backup job: use correct default value for pbsUserSelector * ui: tape: backup job: use correct default value for pbsUserSelector
@ -774,22 +93,7 @@ rust-proxmox-backup (2.0.0-1) bullseye; urgency=medium
* backup: add helpers for async last recently used (LRU) caches for chunk * backup: add helpers for async last recently used (LRU) caches for chunk
and index reading of backup snapshot and index reading of backup snapshot
* fix #3459: manager: add --ignore-verified and --outdated-after parameters -- Proxmox Support Team <support@proxmox.com> Wed, 16 Jun 2021 09:46:15 +0200
* proxmox-backup-manager: show task log on datastore create
* tape: snapshot reader: read chunks sorted by inode (per index) to improve
sequential reads when backing up data from slow spinning disks to tape.
* file-restore: support ZFS pools
* improve fix for #3393: pxar create: try to read xattrs/fcaps/acls by default
* fix compatibility with ExtJS 7.0
* docs: build api-viewer from widget-toolkit-dev
-- Proxmox Support Team <support@proxmox.com> Mon, 28 Jun 2021 19:35:40 +0200
rust-proxmox-backup (1.1.9-1) stable; urgency=medium rust-proxmox-backup (1.1.9-1) stable; urgency=medium

98
debian/control vendored
View File

@ -1,84 +1,62 @@
Source: rust-proxmox-backup Source: rust-proxmox-backup
Section: admin Section: admin
Priority: optional Priority: optional
Build-Depends: debhelper (>= 12), Build-Depends: debhelper (>= 11),
dh-cargo (>= 24), dh-cargo (>= 18),
cargo:native, cargo:native,
rustc:native, rustc:native,
libstd-rust-dev, libstd-rust-dev,
librust-anyhow-1+default-dev, librust-anyhow-1+default-dev,
librust-apt-pkg-native-0.3+default-dev (>= 0.3.2-~~), librust-apt-pkg-native-0.3+default-dev (>= 0.3.2-~~),
librust-base64-0.13+default-dev, librust-base64-0.12+default-dev,
librust-bitflags-1+default-dev (>= 1.2.1-~~), librust-bitflags-1+default-dev (>= 1.2.1-~~),
librust-bytes-1+default-dev, librust-bytes-1+default-dev,
librust-cidr-0.2+default-dev (>= 0.2.1-~~),
librust-crc32fast-1+default-dev, librust-crc32fast-1+default-dev,
librust-crossbeam-channel-0.5+default-dev, librust-crossbeam-channel-0.5+default-dev,
librust-endian-trait-0.6+arrays-dev, librust-endian-trait-0.6+arrays-dev,
librust-endian-trait-0.6+default-dev, librust-endian-trait-0.6+default-dev,
librust-env-logger-0.9+default-dev, librust-env-logger-0.7+default-dev,
librust-flate2-1+default-dev, librust-flate2-1+default-dev,
librust-foreign-types-0.3+default-dev, librust-foreign-types-0.3+default-dev,
librust-futures-0.3+default-dev, librust-futures-0.3+default-dev,
librust-h2-0.3+default-dev, librust-h2-0.3+default-dev,
librust-h2-0.3+stream-dev, librust-h2-0.3+stream-dev,
librust-handlebars-3+default-dev, librust-handlebars-3+default-dev,
librust-hex-0.4+default-dev (>= 0.4.3-~~),
librust-hex-0.4+serde-dev (>= 0.4.3-~~),
librust-http-0.2+default-dev, librust-http-0.2+default-dev,
librust-hyper-0.14+default-dev (>= 0.14.5-~~), librust-hyper-0.14+default-dev,
librust-hyper-0.14+full-dev (>= 0.14.5-~~), librust-hyper-0.14+full-dev,
librust-lazy-static-1+default-dev (>= 1.4-~~), librust-lazy-static-1+default-dev (>= 1.4-~~),
librust-libc-0.2+default-dev, librust-libc-0.2+default-dev,
librust-log-0.4+default-dev (>= 0.4.17-~~) <!nocheck>, librust-log-0.4+default-dev,
librust-nix-0.24+default-dev, librust-nix-0.19+default-dev (>= 0.19.1-~~),
librust-nom-5+default-dev (>= 5.1-~~), librust-nom-5+default-dev (>= 5.1-~~),
librust-num-traits-0.2+default-dev, librust-num-traits-0.2+default-dev,
librust-once-cell-1+default-dev (>= 1.3.1-~~), librust-once-cell-1+default-dev (>= 1.3.1-~~),
librust-openssl-0.10+default-dev (>= 0.10.38-~~), librust-openssl-0.10+default-dev,
librust-pam-0.7+default-dev, librust-pam-0.7+default-dev,
librust-pam-sys-0.5+default-dev, librust-pam-sys-0.5+default-dev,
librust-pathpatterns-0.1+default-dev (>= 0.1.2-~~), librust-pathpatterns-0.1+default-dev (>= 0.1.2-~~),
librust-percent-encoding-2+default-dev (>= 2.1-~~), librust-percent-encoding-2+default-dev (>= 2.1-~~),
librust-pin-project-lite-0.2+default-dev, librust-pin-project-1+default-dev,
librust-proxmox-acme-rs-0.4+default-dev, librust-pin-utils-0.1+default-dev,
librust-proxmox-apt-0.8+default-dev, librust-proxmox-0.11+api-macro-dev (>= 0.11.6-~~),
librust-proxmox-async-0.4+default-dev, librust-proxmox-0.11+cli-dev (>= 0.11.6-~~),
librust-proxmox-borrow-1+default-dev, librust-proxmox-0.11+default-dev (>= 0.11.6-~~),
librust-proxmox-compression-0.1+default-dev (>= 0.1.1-~~), librust-proxmox-0.11+router-dev (>= 0.11.6-~~),
librust-proxmox-0.11+sortable-macro-dev (>= 0.11.6-~~),
librust-proxmox-0.11+tfa-dev (>= 0.11.6-~~),
librust-proxmox-acme-rs-0.3+default-dev,
librust-proxmox-fuse-0.1+default-dev (>= 0.1.1-~~), librust-proxmox-fuse-0.1+default-dev (>= 0.1.1-~~),
librust-proxmox-http-0.6+client-dev (>= 0.6.1-~~), librust-proxmox-http-0.2+client-dev (>= 0.2.1-~~),
librust-proxmox-http-0.6+default-dev (>= 0.6.1-~~), librust-proxmox-http-0.2+default-dev (>= 0.2.1-~~),
librust-proxmox-http-0.6+http-helpers-dev (>= 0.6.1-~~), librust-proxmox-http-0.2+http-helpers-dev (>= 0.2.1-~~),
librust-proxmox-http-0.6+websocket-dev (>= 0.6.1-~~), librust-proxmox-http-0.2+websocket-dev (>= 0.2.1-~~),
librust-proxmox-io-1+default-dev (>= 1.0.1-~~),
librust-proxmox-io-1+tokio-dev (>= 1.0.1-~~),
librust-proxmox-lang-1+default-dev (>= 1.1-~~),
librust-proxmox-openid-0.9+default-dev,
librust-proxmox-router-1+cli-dev (>= 1.2-~~),
librust-proxmox-router-1+default-dev (>= 1.2.2-~~),
librust-proxmox-schema-1+api-macro-dev (>= 1.3.1-~~),
librust-proxmox-schema-1+default-dev (>= 1.3.1-~~),
librust-proxmox-schema-1+upid-api-impl-dev (>= 1.3.1-~~),
librust-proxmox-section-config-1+default-dev,
librust-proxmox-serde-0.1+default-dev,
librust-proxmox-shared-memory-0.2+default-dev,
librust-proxmox-sys-0.3+default-dev,
librust-proxmox-sys-0.3+logrotate-dev,
librust-proxmox-sys-0.3+sortable-macro-dev,
librust-proxmox-tfa-2+api-dev,
librust-proxmox-tfa-2+api-types-dev,
librust-proxmox-tfa-2+default-dev,
librust-proxmox-time-1+default-dev (>= 1.1.2-~~),
librust-proxmox-uuid-1+default-dev,
librust-proxmox-uuid-1+serde-dev,
librust-pxar-0.10+default-dev (>= 0.10.1-~~), librust-pxar-0.10+default-dev (>= 0.10.1-~~),
librust-pxar-0.10+tokio-io-dev (>= 0.10.1-~~), librust-pxar-0.10+tokio-io-dev (>= 0.10.1-~~),
librust-regex-1+default-dev (>= 1.5.5-~~), librust-regex-1+default-dev (>= 1.2-~~),
librust-rustyline-9+default-dev, librust-rustyline-7+default-dev,
librust-serde-1+default-dev, librust-serde-1+default-dev,
librust-serde-1+derive-dev, librust-serde-1+derive-dev,
librust-serde-cbor-0.11+default-dev (>= 0.11.1-~~),
librust-serde-json-1+default-dev, librust-serde-json-1+default-dev,
librust-siphasher-0.3+default-dev, librust-siphasher-0.3+default-dev,
librust-syslog-4+default-dev, librust-syslog-4+default-dev,
@ -94,23 +72,23 @@ Build-Depends: debhelper (>= 12),
librust-tokio-1+rt-dev (>= 1.6-~~), librust-tokio-1+rt-dev (>= 1.6-~~),
librust-tokio-1+rt-multi-thread-dev (>= 1.6-~~), librust-tokio-1+rt-multi-thread-dev (>= 1.6-~~),
librust-tokio-1+signal-dev (>= 1.6-~~), librust-tokio-1+signal-dev (>= 1.6-~~),
librust-tokio-1+sync-dev (>= 1.6-~~),
librust-tokio-1+time-dev (>= 1.6-~~), librust-tokio-1+time-dev (>= 1.6-~~),
librust-tokio-openssl-0.6+default-dev (>= 0.6.1-~~), librust-tokio-openssl-0.6+default-dev (>= 0.6.1-~~),
librust-tokio-stream-0.1+default-dev, librust-tokio-stream-0.1+default-dev,
librust-tokio-util-0.7+codec-dev, librust-tokio-util-0.6+codec-dev,
librust-tokio-util-0.7+default-dev, librust-tokio-util-0.6+default-dev,
librust-tokio-util-0.7+io-dev, librust-tokio-util-0.6+io-dev,
librust-tower-service-0.3+default-dev, librust-tower-service-0.3+default-dev,
librust-udev-0.4+default-dev, librust-udev-0.4+default-dev | librust-udev-0.3+default-dev,
librust-url-2+default-dev (>= 2.1-~~), librust-url-2+default-dev (>= 2.1-~~),
librust-walkdir-2+default-dev, librust-walkdir-2+default-dev,
librust-webauthn-rs-0.2+default-dev (>= 0.2.5-~~),
librust-xdg-2+default-dev (>= 2.2-~~), librust-xdg-2+default-dev (>= 2.2-~~),
librust-zstd-0.6+bindgen-dev, librust-zstd-0.4+bindgen-dev,
librust-zstd-0.6+default-dev, librust-zstd-0.4+default-dev,
libacl1-dev, libacl1-dev,
libfuse3-dev, libfuse3-dev,
libsystemd-dev (>= 246-~~), libsystemd-dev,
uuid-dev, uuid-dev,
libsgutils2-dev, libsgutils2-dev,
bash-completion, bash-completion,
@ -121,7 +99,6 @@ Build-Depends: debhelper (>= 12),
graphviz <!nodoc>, graphviz <!nodoc>,
latexmk <!nodoc>, latexmk <!nodoc>,
patchelf, patchelf,
proxmox-widget-toolkit-dev <!nodoc>,
pve-eslint (>= 7.18.0-1), pve-eslint (>= 7.18.0-1),
python3-docutils, python3-docutils,
python3-pygments, python3-pygments,
@ -132,16 +109,15 @@ Build-Depends: debhelper (>= 12),
texlive-xetex <!nodoc>, texlive-xetex <!nodoc>,
xindy <!nodoc> xindy <!nodoc>
Maintainer: Proxmox Support Team <support@proxmox.com> Maintainer: Proxmox Support Team <support@proxmox.com>
Standards-Version: 4.5.1 Standards-Version: 4.4.1
Vcs-Git: git://git.proxmox.com/git/proxmox-backup.git Vcs-Git: git://git.proxmox.com/git/proxmox-backup.git
Vcs-Browser: https://git.proxmox.com/?p=proxmox-backup.git;a=summary Vcs-Browser: https://git.proxmox.com/?p=proxmox-backup.git;a=summary
Homepage: https://www.proxmox.com Homepage: https://www.proxmox.com
Rules-Requires-Root: binary-targets
Package: proxmox-backup-server Package: proxmox-backup-server
Architecture: any Architecture: any
Depends: fonts-font-awesome, Depends: fonts-font-awesome,
libjs-extjs (>= 7~), libjs-extjs (>= 6.0.1),
libjs-qrcodejs (>= 1.20201119), libjs-qrcodejs (>= 1.20201119),
libproxmox-acme-plugins, libproxmox-acme-plugins,
libsgutils2-2, libsgutils2-2,
@ -152,7 +128,7 @@ Depends: fonts-font-awesome,
postfix | mail-transport-agent, postfix | mail-transport-agent,
proxmox-backup-docs, proxmox-backup-docs,
proxmox-mini-journalreader, proxmox-mini-journalreader,
proxmox-widget-toolkit (>= 3.4-3), proxmox-widget-toolkit (>= 2.6-2),
pve-xtermjs (>= 4.7.0-1), pve-xtermjs (>= 4.7.0-1),
sg3-utils, sg3-utils,
smartmontools, smartmontools,
@ -176,8 +152,7 @@ Description: Proxmox Backup Client tools
Package: proxmox-backup-docs Package: proxmox-backup-docs
Build-Profiles: <!nodoc> Build-Profiles: <!nodoc>
Section: doc Section: doc
Depends: fonts-font-awesome, Depends: libjs-extjs,
libjs-extjs,
libjs-mathjax, libjs-mathjax,
${misc:Depends}, ${misc:Depends},
Architecture: all Architecture: all
@ -190,7 +165,6 @@ Depends: ${misc:Depends},
${shlibs:Depends}, ${shlibs:Depends},
Recommends: pve-qemu-kvm (>= 5.0.0-9), Recommends: pve-qemu-kvm (>= 5.0.0-9),
proxmox-backup-restore-image, proxmox-backup-restore-image,
Breaks: proxmox-backup-restore-image (<< 0.3.1)
Description: Proxmox Backup single file restore tools for pxar and block device backups Description: Proxmox Backup single file restore tools for pxar and block device backups
This package contains the Proxmox Backup single file restore client for This package contains the Proxmox Backup single file restore client for
restoring individual files and folders from both host/container and VM/block restoring individual files and folders from both host/container and VM/block

55
debian/control.in vendored Normal file
View File

@ -0,0 +1,55 @@
Package: proxmox-backup-server
Architecture: any
Depends: fonts-font-awesome,
libjs-extjs (>= 6.0.1),
libjs-qrcodejs (>= 1.20201119),
libproxmox-acme-plugins,
libsgutils2-2,
libzstd1 (>= 1.3.8),
lvm2,
openssh-server,
pbs-i18n,
postfix | mail-transport-agent,
proxmox-backup-docs,
proxmox-mini-journalreader,
proxmox-widget-toolkit (>= 2.6-2),
pve-xtermjs (>= 4.7.0-1),
sg3-utils,
smartmontools,
${misc:Depends},
${shlibs:Depends},
Recommends: zfsutils-linux,
ifupdown2,
Description: Proxmox Backup Server daemon with tools and GUI
This package contains the Proxmox Backup Server daemons and related
tools. This includes a web-based graphical user interface.
Package: proxmox-backup-client
Architecture: any
Depends: qrencode,
${misc:Depends},
${shlibs:Depends},
Description: Proxmox Backup Client tools
This package contains the Proxmox Backup client, which provides a
simple command line tool to create and restore backups.
Package: proxmox-backup-docs
Build-Profiles: <!nodoc>
Section: doc
Depends: libjs-extjs,
libjs-mathjax,
${misc:Depends},
Architecture: all
Description: Proxmox Backup Documentation
This package contains the Proxmox Backup Documentation files.
Package: proxmox-backup-file-restore
Architecture: any
Depends: ${misc:Depends},
${shlibs:Depends},
Recommends: pve-qemu-kvm (>= 5.0.0-9),
proxmox-backup-restore-image,
Description: Proxmox Backup single file restore tools for pxar and block device backups
This package contains the Proxmox Backup single file restore client for
restoring individual files and folders from both host/container and VM/block
device backups. It includes a block device restore driver using QEMU.

42
debian/debcargo.toml vendored Normal file
View File

@ -0,0 +1,42 @@
overlay = "."
crate_src_path = ".."
whitelist = ["tests/*.c"]
maintainer = "Proxmox Support Team <support@proxmox.com>"
[source]
vcs_git = "git://git.proxmox.com/git/proxmox-backup.git"
vcs_browser = "https://git.proxmox.com/?p=proxmox-backup.git;a=summary"
section = "admin"
build_depends = [
"bash-completion",
"debhelper (>= 12~)",
"fonts-dejavu-core <!nodoc>",
"fonts-lato <!nodoc>",
"fonts-open-sans <!nodoc>",
"graphviz <!nodoc>",
"latexmk <!nodoc>",
"patchelf",
"pve-eslint (>= 7.18.0-1)",
"python3-docutils",
"python3-pygments",
"python3-sphinx <!nodoc>",
"rsync",
"texlive-fonts-extra <!nodoc>",
"texlive-fonts-recommended <!nodoc>",
"texlive-xetex <!nodoc>",
"xindy <!nodoc>",
]
build_depends_excludes = [
"debhelper (>=11)",
]
[packages.lib]
depends = [
"libacl1-dev",
"libfuse3-dev",
"libsystemd-dev",
"uuid-dev",
"libsgutils2-dev",
]

74
debian/postinst vendored
View File

@ -4,14 +4,6 @@ set -e
#DEBHELPER# #DEBHELPER#
update_sync_job() {
job="$1"
echo "Updating sync job '$job' to make old 'remove-vanished' default explicit.."
proxmox-backup-manager sync-job update "$job" --remove-vanished true \
|| echo "Failed, please check sync.cfg manually!"
}
case "$1" in case "$1" in
configure) configure)
# need to have user backup in the tape group # need to have user backup in the tape group
@ -34,42 +26,48 @@ case "$1" in
fi fi
deb-systemd-invoke $_dh_action proxmox-backup.service proxmox-backup-proxy.service >/dev/null || true deb-systemd-invoke $_dh_action proxmox-backup.service proxmox-backup-proxy.service >/dev/null || true
# FIXME: Remove with 1.1
if test -n "$2"; then if test -n "$2"; then
if dpkg --compare-versions "$2" 'lt' '0.9.4-1'; then
if grep -s -q -P -e '^\s+verify-schedule ' /etc/proxmox-backup/datastore.cfg; then
echo "NOTE: drop all verify schedules from datastore config."
echo "You can now add more flexible verify jobs"
flock -w 30 /etc/proxmox-backup/.datastore.lck \
sed -i '/^\s\+verify-schedule /d' /etc/proxmox-backup/datastore.cfg || true
fi
fi
if dpkg --compare-versions "$2" 'le' '0.9.5-1'; then
chown --quiet backup:backup /var/log/proxmox-backup/api/auth.log || true
fi
if dpkg --compare-versions "$2" 'le' '0.9.7-1'; then
if [ -e /etc/proxmox-backup/remote.cfg ]; then
echo "NOTE: Switching over remote.cfg to new field names.."
flock -w 30 /etc/proxmox-backup/.remote.lck \
sed -i \
-e 's/^\s\+userid /\tauth-id /g' \
/etc/proxmox-backup/remote.cfg || true
fi
fi
if dpkg --compare-versions "$2" 'le' '1.0.14-1'; then
# FIXME: Remove with 2.0
if grep -s -q -P -e '^linux:' /etc/proxmox-backup/tape.cfg; then
echo "========="
echo "= NOTE: You have now unsupported 'linux' tape drives configured."
echo "= * Execute 'udevadm control --reload-rules && udevadm trigger' to update /dev"
echo "= * Edit '/etc/proxmox-backup/tape.cfg', remove 'linux' entries and re-add over CLI/GUI"
echo "========="
fi
fi
# FIXME: remove with 2.0
if [ -d "/var/lib/proxmox-backup/tape" ] &&
[ "$(stat --printf '%a' '/var/lib/proxmox-backup/tape')" != "750" ]; then
chmod 0750 /var/lib/proxmox-backup/tape || true
fi
# FIXME: Remove in future version once we're sure no broken entries remain in anyone's files # FIXME: Remove in future version once we're sure no broken entries remain in anyone's files
if grep -q -e ':termproxy::[^@]\+: ' /var/log/proxmox-backup/tasks/active; then if grep -q -e ':termproxy::[^@]\+: ' /var/log/proxmox-backup/tasks/active; then
echo "Fixing up termproxy user id in task log..." echo "Fixing up termproxy user id in task log..."
flock -w 30 /var/log/proxmox-backup/tasks/active.lock sed -i 's/:termproxy::\([^@]\+\): /:termproxy::\1@pam: /' /var/log/proxmox-backup/tasks/active || true flock -w 30 /var/log/proxmox-backup/tasks/active.lock sed -i 's/:termproxy::\([^@]\+\): /:termproxy::\1@pam: /' /var/log/proxmox-backup/tasks/active || true
fi fi
if dpkg --compare-versions "$2" 'lt' '2.2.2~'; then
echo "moving prune schedule from datacenter config to new prune job config"
proxmox-backup-manager update-to-prune-jobs-config \
|| echo "Failed to move prune jobs, please check manually"
true
fi
if dpkg --compare-versions "$2" 'lt' '2.1.3~' && test -e /etc/proxmox-backup/sync.cfg; then
prev_job=""
# read from HERE doc because POSIX sh limitations
while read -r key value; do
if test "$key" = "sync:"; then
if test -n "$prev_job"; then
# previous job doesn't have an explicit value
update_sync_job "$prev_job"
fi
prev_job=$value
else
prev_job=""
fi
done <<EOF
$(grep -e '^sync:' -e 'remove-vanished' /etc/proxmox-backup/sync.cfg)
EOF
if test -n "$prev_job"; then
# last job doesn't have an explicit value
update_sync_job "$prev_job"
fi
fi
fi fi
;; ;;

View File

@ -1,8 +0,0 @@
# proxmox-backup-debug bash completion
# see http://tiswww.case.edu/php/chet/bash/FAQ
# and __ltrim_colon_completions() in /usr/share/bash-completion/bash_completion
# this modifies global var, but I found no better way
COMP_WORDBREAKS=${COMP_WORDBREAKS//:}
complete -C 'proxmox-backup-debug bashcomplete' proxmox-backup-debug

View File

@ -1,6 +1,5 @@
/usr/share/doc/proxmox-backup/proxmox-backup.pdf /usr/share/doc/proxmox-backup/html/proxmox-backup.pdf /usr/share/doc/proxmox-backup/proxmox-backup.pdf /usr/share/doc/proxmox-backup/html/proxmox-backup.pdf
/usr/share/javascript/extjs /usr/share/doc/proxmox-backup/html/prune-simulator/extjs /usr/share/javascript/extjs /usr/share/doc/proxmox-backup/html/prune-simulator/extjs
/usr/share/javascript/extjs /usr/share/doc/proxmox-backup/html/lto-barcode/extjs /usr/share/javascript/extjs /usr/share/doc/proxmox-backup/html/lto-barcode/extjs
/usr/share/fonts-font-awesome/ /usr/share/doc/proxmox-backup/html/lto-barcode/font-awesome
/usr/share/javascript/extjs /usr/share/doc/proxmox-backup/html/api-viewer/extjs /usr/share/javascript/extjs /usr/share/doc/proxmox-backup/html/api-viewer/extjs
/usr/share/javascript/mathjax /usr/share/doc/proxmox-backup/html/_static/mathjax /usr/share/javascript/mathjax /usr/share/doc/proxmox-backup/html/_static/mathjax

View File

@ -1,5 +1,4 @@
debian/proxmox-backup-manager.bc proxmox-backup-manager debian/proxmox-backup-manager.bc proxmox-backup-manager
debian/proxmox-backup-debug.bc proxmox-backup-debug
debian/proxmox-tape.bc proxmox-tape debian/proxmox-tape.bc proxmox-tape
debian/pmtx.bc pmtx debian/pmtx.bc pmtx
debian/pmt.bc pmt debian/pmt.bc pmt

View File

@ -9,7 +9,6 @@ usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-backup-proxy
usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-backup-banner usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-backup-banner
usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-daily-update usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-daily-update
usr/lib/x86_64-linux-gnu/proxmox-backup/sg-tape-cmd usr/lib/x86_64-linux-gnu/proxmox-backup/sg-tape-cmd
usr/sbin/proxmox-backup-debug
usr/sbin/proxmox-backup-manager usr/sbin/proxmox-backup-manager
usr/bin/pmtx usr/bin/pmtx
usr/bin/pmt usr/bin/pmt
@ -18,7 +17,6 @@ usr/share/javascript/proxmox-backup/index.hbs
usr/share/javascript/proxmox-backup/css/ext6-pbs.css usr/share/javascript/proxmox-backup/css/ext6-pbs.css
usr/share/javascript/proxmox-backup/images usr/share/javascript/proxmox-backup/images
usr/share/javascript/proxmox-backup/js/proxmox-backup-gui.js usr/share/javascript/proxmox-backup/js/proxmox-backup-gui.js
usr/share/man/man1/proxmox-backup-debug.1
usr/share/man/man1/proxmox-backup-manager.1 usr/share/man/man1/proxmox-backup-manager.1
usr/share/man/man1/proxmox-backup-proxy.1 usr/share/man/man1/proxmox-backup-proxy.1
usr/share/man/man1/proxmox-tape.1 usr/share/man/man1/proxmox-tape.1
@ -33,7 +31,6 @@ usr/share/man/man5/verification.cfg.5
usr/share/man/man5/media-pool.cfg.5 usr/share/man/man5/media-pool.cfg.5
usr/share/man/man5/tape.cfg.5 usr/share/man/man5/tape.cfg.5
usr/share/man/man5/tape-job.cfg.5 usr/share/man/man5/tape-job.cfg.5
usr/share/zsh/vendor-completions/_proxmox-backup-debug
usr/share/zsh/vendor-completions/_proxmox-backup-manager usr/share/zsh/vendor-completions/_proxmox-backup-manager
usr/share/zsh/vendor-completions/_proxmox-tape usr/share/zsh/vendor-completions/_proxmox-tape
usr/share/zsh/vendor-completions/_pmtx usr/share/zsh/vendor-completions/_pmtx

5
debian/rules vendored
View File

@ -45,6 +45,11 @@ override_dh_installsystemd:
override_dh_fixperms: override_dh_fixperms:
dh_fixperms --exclude sg-tape-cmd dh_fixperms --exclude sg-tape-cmd
# workaround https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=933541
# TODO: remove once available (Debian 11 ?)
override_dh_dwz:
dh_dwz --no-dwz-multifile
override_dh_strip: override_dh_strip:
dh_strip dh_strip
for exe in $$(find \ for exe in $$(find \

View File

@ -5,7 +5,6 @@ GENERATED_SYNOPSIS := \
proxmox-backup-client/synopsis.rst \ proxmox-backup-client/synopsis.rst \
proxmox-backup-client/catalog-shell-synopsis.rst \ proxmox-backup-client/catalog-shell-synopsis.rst \
proxmox-backup-manager/synopsis.rst \ proxmox-backup-manager/synopsis.rst \
proxmox-backup-debug/synopsis.rst \
proxmox-file-restore/synopsis.rst \ proxmox-file-restore/synopsis.rst \
pxar/synopsis.rst \ pxar/synopsis.rst \
pmtx/synopsis.rst \ pmtx/synopsis.rst \
@ -28,8 +27,7 @@ MAN1_PAGES := \
proxmox-backup-proxy.1 \ proxmox-backup-proxy.1 \
proxmox-backup-client.1 \ proxmox-backup-client.1 \
proxmox-backup-manager.1 \ proxmox-backup-manager.1 \
proxmox-file-restore.1 \ proxmox-file-restore.1
proxmox-backup-debug.1
MAN5_PAGES := \ MAN5_PAGES := \
media-pool.cfg.5 \ media-pool.cfg.5 \
@ -48,35 +46,23 @@ PRUNE_SIMULATOR_FILES := \
prune-simulator/clear-trigger.png \ prune-simulator/clear-trigger.png \
prune-simulator/prune-simulator.js prune-simulator/prune-simulator.js
PRUNE_SIMULATOR_JS_SOURCE := \
/usr/share/javascript/proxmox-widget-toolkit-dev/Toolkit.js \
prune-simulator/prune-simulator_source.js
LTO_BARCODE_JS_SOURCE := \
/usr/share/javascript/proxmox-widget-toolkit-dev/Toolkit.js \
lto-barcode/code39.js \
lto-barcode/prefix-field.js \
lto-barcode/label-style.js \
lto-barcode/tape-type.js \
lto-barcode/paper-size.js \
lto-barcode/page-layout.js \
lto-barcode/page-calibration.js \
lto-barcode/label-list.js \
lto-barcode/label-setup.js \
lto-barcode/lto-barcode.js
LTO_BARCODE_FILES := \ LTO_BARCODE_FILES := \
lto-barcode/index.html \ lto-barcode/index.html \
lto-barcode/lto-barcode-generator.js lto-barcode/code39.js \
lto-barcode/prefix-field.js \
lto-barcode/label-style.js \
lto-barcode/tape-type.js \
lto-barcode/paper-size.js \
lto-barcode/page-layout.js \
lto-barcode/page-calibration.js \
lto-barcode/label-list.js \
lto-barcode/label-setup.js \
lto-barcode/lto-barcode.js
API_VIEWER_SOURCES= \ API_VIEWER_SOURCES= \
api-viewer/index.html \ api-viewer/index.html \
api-viewer/apidoc.js api-viewer/apidoc.js
API_VIEWER_FILES := \
api-viewer/apidata.js \
/usr/share/javascript/proxmox-widget-toolkit-dev/APIViewer.js \
# Sphinx documentation setup # Sphinx documentation setup
SPHINXOPTS = SPHINXOPTS =
SPHINXBUILD = sphinx-build SPHINXBUILD = sphinx-build
@ -201,12 +187,6 @@ proxmox-file-restore/synopsis.rst: ${COMPILEDIR}/proxmox-file-restore
proxmox-file-restore.1: proxmox-file-restore/man1.rst proxmox-file-restore/description.rst proxmox-file-restore/synopsis.rst proxmox-file-restore.1: proxmox-file-restore/man1.rst proxmox-file-restore/description.rst proxmox-file-restore/synopsis.rst
rst2man $< >$@ rst2man $< >$@
proxmox-backup-debug/synopsis.rst: ${COMPILEDIR}/proxmox-backup-debug
${COMPILEDIR}/proxmox-backup-debug printdoc > proxmox-backup-debug/synopsis.rst
proxmox-backup-debug.1: proxmox-backup-debug/man1.rst proxmox-backup-debug/description.rst proxmox-backup-debug/synopsis.rst
rst2man $< >$@
.PHONY: onlinehelpinfo .PHONY: onlinehelpinfo
onlinehelpinfo: onlinehelpinfo:
@echo "Generating OnlineHelpInfo.js..." @echo "Generating OnlineHelpInfo.js..."
@ -216,17 +196,8 @@ onlinehelpinfo:
api-viewer/apidata.js: ${COMPILEDIR}/docgen api-viewer/apidata.js: ${COMPILEDIR}/docgen
${COMPILEDIR}/docgen apidata.js >$@ ${COMPILEDIR}/docgen apidata.js >$@
api-viewer/apidoc.js: ${API_VIEWER_FILES} api-viewer/apidoc.js: api-viewer/apidata.js api-viewer/PBSAPI.js
cat ${API_VIEWER_FILES} >$@.tmp cat api-viewer/apidata.js api-viewer/PBSAPI.js >$@
mv $@.tmp $@
prune-simulator/prune-simulator.js: ${PRUNE_SIMULATOR_JS_SOURCE}
cat ${PRUNE_SIMULATOR_JS_SOURCE} >$@.tmp
mv $@.tmp $@
lto-barcode/lto-barcode-generator.js: ${LTO_BARCODE_JS_SOURCE}
cat ${LTO_BARCODE_JS_SOURCE} >$@.tmp
mv $@.tmp $@
.PHONY: html .PHONY: html
html: ${GENERATED_SYNOPSIS} images/proxmox-logo.svg custom.css conf.py ${PRUNE_SIMULATOR_FILES} ${LTO_BARCODE_FILES} ${API_VIEWER_SOURCES} html: ${GENERATED_SYNOPSIS} images/proxmox-logo.svg custom.css conf.py ${PRUNE_SIMULATOR_FILES} ${LTO_BARCODE_FILES} ${API_VIEWER_SOURCES}
@ -257,7 +228,7 @@ epub3: ${GENERATED_SYNOPSIS}
clean: clean:
rm -r -f *~ *.1 ${BUILDDIR} ${GENERATED_SYNOPSIS} api-viewer/apidata.js rm -r -f *~ *.1 ${BUILDDIR} ${GENERATED_SYNOPSIS} api-viewer/apidata.js
rm -f api-viewer/apidoc.js lto-barcode/lto-barcode-generator.js prune-simulator/prune-simulator.js rm -f api-viewer/apidoc.js lto-barcode/lto-barcode-generator.js
install_manual_pages: ${MAN1_PAGES} ${MAN5_PAGES} install_manual_pages: ${MAN1_PAGES} ${MAN5_PAGES}

526
docs/api-viewer/PBSAPI.js Normal file
View File

@ -0,0 +1,526 @@
// avoid errors when running without development tools
if (!Ext.isDefined(Ext.global.console)) {
var console = {
dir: function() {},
log: function() {}
};
}
Ext.onReady(function() {
Ext.define('pve-param-schema', {
extend: 'Ext.data.Model',
fields: [
'name', 'type', 'typetext', 'description', 'verbose_description',
'enum', 'minimum', 'maximum', 'minLength', 'maxLength',
'pattern', 'title', 'requires', 'format', 'default',
'disallow', 'extends', 'links',
{
name: 'optional',
type: 'boolean'
}
]
});
var store = Ext.define('pve-updated-treestore', {
extend: 'Ext.data.TreeStore',
model: Ext.define('pve-api-doc', {
extend: 'Ext.data.Model',
fields: [
'path', 'info', 'text',
]
}),
proxy: {
type: 'memory',
data: pbsapi
},
sorters: [{
property: 'leaf',
direction: 'ASC'
}, {
property: 'text',
direction: 'ASC'
}],
filterer: 'bottomup',
doFilter: function(node) {
this.filterNodes(node, this.getFilters().getFilterFn(), true);
},
filterNodes: function(node, filterFn, parentVisible) {
var me = this,
bottomUpFiltering = me.filterer === 'bottomup',
match = filterFn(node) && parentVisible || (node.isRoot() && !me.getRootVisible()),
childNodes = node.childNodes,
len = childNodes && childNodes.length, i, matchingChildren;
if (len) {
for (i = 0; i < len; ++i) {
matchingChildren = me.filterNodes(childNodes[i], filterFn, match || bottomUpFiltering) || matchingChildren;
}
if (bottomUpFiltering) {
match = matchingChildren || match;
}
}
node.set("visible", match, me._silentOptions);
return match;
},
}).create();
var render_description = function(value, metaData, record) {
var pdef = record.data;
value = pdef.verbose_description || value;
// TODO: try to render asciidoc correctly
metaData.style = 'white-space:pre-wrap;'
return Ext.htmlEncode(value);
};
var render_type = function(value, metaData, record) {
var pdef = record.data;
return pdef['enum'] ? 'enum' : (pdef.type || 'string');
};
let render_simple_format = function(pdef, type_fallback) {
if (pdef.typetext)
return pdef.typetext;
if (pdef['enum'])
return pdef['enum'].join(' | ');
if (pdef.format)
return pdef.format;
if (pdef.pattern)
return pdef.pattern;
if (pdef.type === 'boolean')
return `<true|false>`;
if (type_fallback && pdef.type)
return `<${pdef.type}>`;
return;
};
let render_format = function(value, metaData, record) {
let pdef = record.data;
metaData.style = 'white-space:normal;'
if (pdef.type === 'array' && pdef.items) {
let format = render_simple_format(pdef.items, true);
return `[${Ext.htmlEncode(format)}, ...]`;
}
return Ext.htmlEncode(render_simple_format(pdef) || '');
};
var real_path = function(path) {
return path.replace(/^.*\/_upgrade_(\/)?/, "/");
};
var permission_text = function(permission) {
let permhtml = "";
if (permission.user) {
if (!permission.description) {
if (permission.user === 'world') {
permhtml += "Accessible without any authentication.";
} else if (permission.user === 'all') {
permhtml += "Accessible by all authenticated users.";
} else {
permhtml += 'Onyl accessible by user "' +
permission.user + '"';
}
}
} else if (permission.check) {
permhtml += "<pre>Check: " +
Ext.htmlEncode(Ext.JSON.encode(permission.check)) + "</pre>";
} else if (permission.userParam) {
permhtml += `<div>Check if user matches parameter '${permission.userParam}'`;
} else if (permission.or) {
permhtml += "<div>Or<div style='padding-left: 10px;'>";
Ext.Array.each(permission.or, function(sub_permission) {
permhtml += permission_text(sub_permission);
})
permhtml += "</div></div>";
} else if (permission.and) {
permhtml += "<div>And<div style='padding-left: 10px;'>";
Ext.Array.each(permission.and, function(sub_permission) {
permhtml += permission_text(sub_permission);
})
permhtml += "</div></div>";
} else {
//console.log(permission);
permhtml += "Unknown syntax!";
}
return permhtml;
};
var render_docu = function(data) {
var md = data.info;
// console.dir(data);
var items = [];
var clicmdhash = {
GET: 'get',
POST: 'create',
PUT: 'set',
DELETE: 'delete'
};
Ext.Array.each(['GET', 'POST', 'PUT', 'DELETE'], function(method) {
var info = md[method];
if (info) {
var usage = "";
usage += "<table><tr><td>HTTP:&nbsp;&nbsp;&nbsp;</td><td>"
+ method + " " + real_path("/api2/json" + data.path) + "</td></tr>";
var sections = [
{
title: 'Description',
html: Ext.htmlEncode(info.description),
bodyPadding: 10
},
{
title: 'Usage',
html: usage,
bodyPadding: 10
}
];
if (info.parameters && info.parameters.properties) {
var pstore = Ext.create('Ext.data.Store', {
model: 'pve-param-schema',
proxy: {
type: 'memory'
},
groupField: 'optional',
sorters: [
{
property: 'name',
direction: 'ASC'
}
]
});
Ext.Object.each(info.parameters.properties, function(name, pdef) {
pdef.name = name;
pstore.add(pdef);
});
pstore.sort();
var groupingFeature = Ext.create('Ext.grid.feature.Grouping',{
enableGroupingMenu: false,
groupHeaderTpl: '<tpl if="groupValue">Optional</tpl><tpl if="!groupValue">Required</tpl>'
});
sections.push({
xtype: 'gridpanel',
title: 'Parameters',
features: [groupingFeature],
store: pstore,
viewConfig: {
trackOver: false,
stripeRows: true
},
columns: [
{
header: 'Name',
dataIndex: 'name',
flex: 1
},
{
header: 'Type',
dataIndex: 'type',
renderer: render_type,
flex: 1
},
{
header: 'Default',
dataIndex: 'default',
flex: 1
},
{
header: 'Format',
dataIndex: 'type',
renderer: render_format,
flex: 2
},
{
header: 'Description',
dataIndex: 'description',
renderer: render_description,
flex: 6
}
]
});
}
if (info.returns) {
var retinf = info.returns;
var rtype = retinf.type;
if (!rtype && retinf.items)
rtype = 'array';
if (!rtype)
rtype = 'object';
var rpstore = Ext.create('Ext.data.Store', {
model: 'pve-param-schema',
proxy: {
type: 'memory'
},
groupField: 'optional',
sorters: [
{
property: 'name',
direction: 'ASC'
}
]
});
var properties;
if (rtype === 'array' && retinf.items.properties) {
properties = retinf.items.properties;
}
if (rtype === 'object' && retinf.properties) {
properties = retinf.properties;
}
Ext.Object.each(properties, function(name, pdef) {
pdef.name = name;
rpstore.add(pdef);
});
rpstore.sort();
var groupingFeature = Ext.create('Ext.grid.feature.Grouping',{
enableGroupingMenu: false,
groupHeaderTpl: '<tpl if="groupValue">Optional</tpl><tpl if="!groupValue">Obligatory</tpl>'
});
var returnhtml;
if (retinf.items) {
returnhtml = '<pre>items: ' + Ext.htmlEncode(JSON.stringify(retinf.items, null, 4)) + '</pre>';
}
if (retinf.properties) {
returnhtml = returnhtml || '';
returnhtml += '<pre>properties:' + Ext.htmlEncode(JSON.stringify(retinf.properties, null, 4)) + '</pre>';
}
var rawSection = Ext.create('Ext.panel.Panel', {
bodyPadding: '0px 10px 10px 10px',
html: returnhtml,
hidden: true
});
sections.push({
xtype: 'gridpanel',
title: 'Returns: ' + rtype,
features: [groupingFeature],
store: rpstore,
viewConfig: {
trackOver: false,
stripeRows: true
},
columns: [
{
header: 'Name',
dataIndex: 'name',
flex: 1
},
{
header: 'Type',
dataIndex: 'type',
renderer: render_type,
flex: 1
},
{
header: 'Default',
dataIndex: 'default',
flex: 1
},
{
header: 'Format',
dataIndex: 'type',
renderer: render_format,
flex: 2
},
{
header: 'Description',
dataIndex: 'description',
renderer: render_description,
flex: 6
}
],
bbar: [
{
xtype: 'button',
text: 'Show RAW',
handler: function(btn) {
rawSection.setVisible(!rawSection.isVisible());
btn.setText(rawSection.isVisible() ? 'Hide RAW' : 'Show RAW');
}}
]
});
sections.push(rawSection);
}
if (!data.path.match(/\/_upgrade_/)) {
var permhtml = '';
if (!info.permissions) {
permhtml = "Root only.";
} else {
if (info.permissions.description) {
permhtml += "<div style='white-space:pre-wrap;padding-bottom:10px;'>" +
Ext.htmlEncode(info.permissions.description) + "</div>";
}
permhtml += permission_text(info.permissions);
}
// we do not have this information for PBS api
//if (!info.allowtoken) {
// permhtml += "<br />This API endpoint is not available for API tokens."
//}
sections.push({
title: 'Required permissions',
bodyPadding: 10,
html: permhtml
});
}
items.push({
title: method,
autoScroll: true,
defaults: {
border: false
},
items: sections
});
}
});
var ct = Ext.getCmp('docview');
ct.setTitle("Path: " + real_path(data.path));
ct.removeAll(true);
ct.add(items);
ct.setActiveTab(0);
};
Ext.define('Ext.form.SearchField', {
extend: 'Ext.form.field.Text',
alias: 'widget.searchfield',
emptyText: 'Search...',
flex: 1,
inputType: 'search',
listeners: {
'change': function(){
var value = this.getValue();
if (!Ext.isEmpty(value)) {
store.filter({
property: 'path',
value: value,
anyMatch: true
});
} else {
store.clearFilter();
}
}
}
});
var tree = Ext.create('Ext.tree.Panel', {
title: 'Resource Tree',
tbar: [
{
xtype: 'searchfield',
}
],
tools: [
{
type: 'expand',
tooltip: 'Expand all',
tooltipType: 'title',
callback: (tree) => tree.expandAll(),
},
{
type: 'collapse',
tooltip: 'Collapse all',
tooltipType: 'title',
callback: (tree) => tree.collapseAll(),
},
],
store: store,
width: 200,
region: 'west',
split: true,
margins: '5 0 5 5',
rootVisible: false,
listeners: {
selectionchange: function(v, selections) {
if (!selections[0])
return;
var rec = selections[0];
render_docu(rec.data);
location.hash = '#' + rec.data.path;
}
}
});
Ext.create('Ext.container.Viewport', {
layout: 'border',
renderTo: Ext.getBody(),
items: [
tree,
{
xtype: 'tabpanel',
title: 'Documentation',
id: 'docview',
region: 'center',
margins: '5 5 5 0',
layout: 'fit',
items: []
}
]
});
var deepLink = function() {
var path = window.location.hash.substring(1).replace(/\/\s*$/, '')
var endpoint = store.findNode('path', path);
if (endpoint) {
tree.getSelectionModel().select(endpoint);
tree.expandPath(endpoint.getPath());
render_docu(endpoint.data);
}
}
window.onhashchange = deepLink;
deepLink();
});

View File

@ -1,33 +1,31 @@
Backup Client Usage Backup Client Usage
=================== ===================
The command line client for Proxmox Backup Server is called The command line client is called :command:`proxmox-backup-client`.
:command:`proxmox-backup-client`.
.. _client_repository: .. _client_repository:
Backup Repository Locations Backup Repository Locations
--------------------------- ---------------------------
The client uses the following format to specify a datastore repository The client uses the following notation to specify a datastore repository
on the backup server (where username is specified in the form of user@realm): on the backup server.
[[username@]server[:port]:]datastore [[username@]server[:port]:]datastore
The default value for ``username`` is ``root@pam``. If no server is specified, The default value for ``username`` is ``root@pam``. If no server is specified,
the default is the local host (``localhost``). the default is the local host (``localhost``).
You can specify a port if your backup server is only reachable on a non-default You can specify a port if your backup server is only reachable on a different
port (for example, with NAT and port forwarding configurations). port (e.g. with NAT and port forwarding).
Note that if the server uses an IPv6 address, you have to write it with square Note that if the server is an IPv6 address, you have to write it with square
brackets (for example, `[fe80::01]`). brackets (for example, `[fe80::01]`).
You can pass the repository with the ``--repository`` command line option, or You can pass the repository with the ``--repository`` command line option, or
by setting the ``PBS_REPOSITORY`` environment variable. by setting the ``PBS_REPOSITORY`` environment variable.
Below are some examples of valid repositories and their corresponding real Here some examples of valid repositories and the real values
values:
================================ ================== ================== =========== ================================ ================== ================== ===========
Example User Host:Port Datastore Example User Host:Port Datastore
@ -48,31 +46,16 @@ Environment Variables
The default backup repository. The default backup repository.
``PBS_PASSWORD`` ``PBS_PASSWORD``
When set, this value is used as the password for the backup server. When set, this value is used for the password required for the backup server.
You can also set this to an API token secret. You can also set this to a API token secret.
``PBS_PASSWORD_FD``, ``PBS_PASSWORD_FILE``, ``PBS_PASSWORD_CMD``
Like ``PBS_PASSWORD``, but read data from an open file descriptor, a file
name or from the `stdout` of a command, respectively. The first defined
environment variable from the order above is preferred.
``PBS_ENCRYPTION_PASSWORD`` ``PBS_ENCRYPTION_PASSWORD``
When set, this value is used to access the secret encryption key (if When set, this value is used to access the secret encryption key (if
protected by password). protected by password).
``PBS_ENCRYPTION_PASSWORD_FD``, ``PBS_ENCRYPTION_PASSWORD_FILE``, ``PBS_ENCRYPTION_PASSWORD_CMD`` ``PBS_FINGERPRINT`` When set, this value is used to verify the server
Like ``PBS_ENCRYPTION_PASSWORD``, but read data from an open file descriptor, certificate (only used if the system CA certificates cannot validate the
a file name or from the `stdout` of a command, respectively. The first certificate).
defined environment variable from the order above is preferred.
``PBS_FINGERPRINT``
When set, this value is used to verify the server certificate (only used if
the system CA certificates cannot validate the certificate).
.. Note:: Passwords must be valid UTF-8 and may not contain newlines. For your
convenience, Proxmox Backup Server only uses the first line as password, so
you can add arbitrary comments after the first newline.
Output Format Output Format
@ -87,15 +70,14 @@ Creating Backups
---------------- ----------------
This section explains how to create a backup from within the machine. This can This section explains how to create a backup from within the machine. This can
be a physical host, a virtual machine, or a container. Such backups may contain be a physical host, a virtual machine, or a container. Such backups may contain file
file and image archives. There are no restrictions in this case. and image archives. There are no restrictions in this case.
.. Note:: If you want to backup virtual machines or containers on Proxmox VE, .. note:: If you want to backup virtual machines or containers on Proxmox VE, see :ref:`pve-integration`.
see :ref:`pve-integration`.
For the following example, you need to have a backup server set up, have working For the following example you need to have a backup server set up, working
credentials, and know the repository name. credentials and need to know the repository name.
In the following examples, we use ``backup-server:store1``. In the following examples we use ``backup-server:store1``.
.. code-block:: console .. code-block:: console
@ -109,32 +91,32 @@ In the following examples, we use ``backup-server:store1``.
Uploaded 12129 chunks in 87 seconds (564 MB/s). Uploaded 12129 chunks in 87 seconds (564 MB/s).
End Time: 2019-12-03T10:36:29+01:00 End Time: 2019-12-03T10:36:29+01:00
This will prompt you for a password, then upload a file archive named This will prompt you for a password and then uploads a file archive named
``root.pxar`` containing all the files in the ``/`` directory. ``root.pxar`` containing all the files in the ``/`` directory.
.. Caution:: Please note that proxmox-backup-client does not .. Caution:: Please note that the proxmox-backup-client does not
automatically include mount points. Instead, you will see a short automatically include mount points. Instead, you will see a short
``skip mount point`` message for each of them. The idea is to ``skip mount point`` notice for each of them. The idea is to
create a separate file archive for each mounted disk. You can create a separate file archive for each mounted disk. You can
explicitly include them using the ``--include-dev`` option explicitly include them using the ``--include-dev`` option
(i.e. ``--include-dev /boot/efi``). You can use this option (i.e. ``--include-dev /boot/efi``). You can use this option
multiple times for each mount point that should be included. multiple times for each mount point that should be included.
The ``--repository`` option can get quite long and is used by all commands. You The ``--repository`` option can get quite long and is used by all
can avoid having to enter this value by setting the environment variable commands. You can avoid having to enter this value by setting the
``PBS_REPOSITORY``. Note that if you would like this to remain set over environment variable ``PBS_REPOSITORY``. Note that if you would like this to remain set
multiple sessions, you should instead add the below line to your ``.bashrc`` over multiple sessions, you should instead add the below line to your
file. ``.bashrc`` file.
.. code-block:: console .. code-block:: console
# export PBS_REPOSITORY=backup-server:store1 # export PBS_REPOSITORY=backup-server:store1
After this, you can execute all commands without having to specify the After this you can execute all commands without specifying the ``--repository``
``--repository`` option. option.
A single backup is allowed to contain more than one archive. For example, if One single backup is allowed to contain more than one archive. For example, if
you want to back up two disks mounted at ``/mnt/disk1`` and ``/mnt/disk2``: you want to backup two disks mounted at ``/mnt/disk1`` and ``/mnt/disk2``:
.. code-block:: console .. code-block:: console
@ -142,71 +124,59 @@ you want to back up two disks mounted at ``/mnt/disk1`` and ``/mnt/disk2``:
This creates a backup of both disks. This creates a backup of both disks.
If you want to use a namespace for the backup target you can add the `--ns` The backup command takes a list of backup specifications, which
parameter: include the archive name on the server, the type of the archive, and the
archive source at the client. The format is:
.. code-block:: console
# proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2 --ns a/b/c
The backup command takes a list of backup specifications, which include the
archive name on the server, the type of the archive, and the archive source at
the client. The format is:
<archive-name>.<type>:<source-path> <archive-name>.<type>:<source-path>
Common types are ``.pxar`` for file archives and ``.img`` for block Common types are ``.pxar`` for file archives, and ``.img`` for block
device images. To create a backup of a block device, run the following command: device images. To create a backup of a block device run the following command:
.. code-block:: console .. code-block:: console
# proxmox-backup-client backup mydata.img:/dev/mylvm/mydata # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata
Excluding Files/Directories from a Backup Excluding files/folders from a backup
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sometimes it is desired to exclude certain files or directories from a backup Sometimes it is desired to exclude certain files or folders from a backup archive.
archive. To tell the Proxmox Backup client when and how to ignore files and To tell the Proxmox Backup client when and how to ignore files and directories,
directories, place a text file named ``.pxarexclude`` in the filesystem place a text file called ``.pxarexclude`` in the filesystem hierarchy.
hierarchy. Whenever the backup client encounters such a file in a directory, Whenever the backup client encounters such a file in a directory, it interprets
it interprets each line as a glob match pattern for files and directories that each line as glob match patterns for files and directories that are to be excluded
are to be excluded from the backup. from the backup.
The file must contain a single glob pattern per line. Empty lines and lines The file must contain a single glob pattern per line. Empty lines are ignored.
starting with ``#`` (indicating a comment) are ignored. The same is true for lines starting with ``#``, which indicates a comment.
A ``!`` at the beginning of a line reverses the glob match pattern from an A ``!`` at the beginning of a line reverses the glob match pattern from an exclusion
exclusion to an explicit inclusion. This makes it possible to exclude all to an explicit inclusion. This makes it possible to exclude all entries in a
entries in a directory except for a few single files/subdirectories. directory except for a few single files/subdirectories.
Lines ending in ``/`` match only on directories. Lines ending in ``/`` match only on directories.
The directory containing the ``.pxarexclude`` file is considered to be the root The directory containing the ``.pxarexclude`` file is considered to be the root of
of the given patterns. It is only possible to match files in this directory and the given patterns. It is only possible to match files in this directory and its subdirectories.
its subdirectories.
.. Note:: Patterns without a leading ``/`` will also match in subdirectories,
while patterns with a leading ``/`` will only match in the current directory.
``\`` is used to escape special glob characters. ``\`` is used to escape special glob characters.
``?`` matches any single character. ``?`` matches any single character.
``*`` matches any character, including an empty string. ``*`` matches any character, including an empty string.
``**`` is used to match current directory and subdirectories. For example, with ``**`` is used to match subdirectories. It can be used to, for example, exclude
the pattern ``**/*.tmp``, it would exclude all files ending in ``.tmp`` within all files ending in ``.tmp`` within the directory or subdirectories with the
a directory and its subdirectories. following pattern ``**/*.tmp``.
``[...]`` matches a single character from any of the provided characters within ``[...]`` matches a single character from any of the provided characters within
the brackets. ``[!...]`` does the complementary and matches any single the brackets. ``[!...]`` does the complementary and matches any single character
character not contained within the brackets. It is also possible to specify not contained within the brackets. It is also possible to specify ranges with two
ranges with two characters separated by ``-``. For example, ``[a-z]`` matches characters separated by ``-``. For example, ``[a-z]`` matches any lowercase
any lowercase alphabetic character, and ``[0-9]`` matches any single digit. alphabetic character and ``[0-9]`` matches any one single digit.
The order of the glob match patterns defines whether a file is included or The order of the glob match patterns defines whether a file is included or
excluded, that is to say, later entries override earlier ones. excluded, that is to say later entries override previous ones.
This is also true for match patterns encountered deeper down the directory This is also true for match patterns encountered deeper down the directory tree,
tree, which can override a previous exclusion. which can override a previous exclusion.
Be aware that excluded directories will **not** be read by the backup client.
.. Note:: Excluded directories will **not** be read by the backup client. Thus, Thus, a ``.pxarexclude`` file in an excluded subdirectory will have no effect.
a ``.pxarexclude`` file in an excluded subdirectory will have no effect. ``.pxarexclude`` files are treated as regular files and will be included in the
``.pxarexclude`` files are treated as regular files and will be included in backup archive.
the backup archive.
For example, consider the following directory structure: For example, consider the following directory structure:
@ -294,7 +264,7 @@ You can avoid entering the passwords by setting the environment
variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``. variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``.
Using a Master Key to Store and Recover Encryption Keys Using a master key to store and recover encryption keys
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can also use ``proxmox-backup-client key`` to create an RSA public/private You can also use ``proxmox-backup-client key`` to create an RSA public/private
@ -374,7 +344,7 @@ To set up a master key:
keep keys ordered and in a place that is separate from the contents being keep keys ordered and in a place that is separate from the contents being
backed up. It can happen, for example, that you back up an entire system, using backed up. It can happen, for example, that you back up an entire system, using
a key on that system. If the system then becomes inaccessible for any reason a key on that system. If the system then becomes inaccessible for any reason
and needs to be restored, this will not be possible, as the encryption key will be and needs to be restored, this will not be possible as the encryption key will be
lost along with the broken system. lost along with the broken system.
It is recommended that you keep your master key safe, but easily accessible, in It is recommended that you keep your master key safe, but easily accessible, in
@ -396,10 +366,10 @@ version of your master key. The following command sends the output of the
Restoring Data Restoring Data
-------------- --------------
The regular creation of backups is a necessary step in avoiding data loss. More The regular creation of backups is a necessary step to avoiding data
importantly, however, is the restoration. It is good practice to perform loss. More importantly, however, is the restoration. It is good practice to perform
periodic recovery tests to ensure that you can access the data in case of periodic recovery tests to ensure that you can access the data in
disaster. case of problems.
First, you need to find the snapshot which you want to restore. The snapshot First, you need to find the snapshot which you want to restore. The snapshot
list command provides a list of all the snapshots on the server: list command provides a list of all the snapshots on the server:
@ -416,11 +386,6 @@ list command provides a list of all the snapshots on the server:
├────────────────────────────────┼─────────────┼────────────────────────────────────┤ ├────────────────────────────────┼─────────────┼────────────────────────────────────┤
... ...
.. tip:: List will by default only output the backup snapshots of the root
namespace itself. To list backups from another namespace use the ``--ns
<ns>`` option
You can inspect the catalog to find specific files. You can inspect the catalog to find specific files.
.. code-block:: console .. code-block:: console
@ -463,22 +428,23 @@ to use the interactive recovery shell.
The interactive recovery shell is a minimal command line interface that The interactive recovery shell is a minimal command line interface that
utilizes the metadata stored in the catalog to quickly list, navigate and utilizes the metadata stored in the catalog to quickly list, navigate and
search for files in a file archive. search files in a file archive.
To restore files, you can select them individually or match them with a glob To restore files, you can select them individually or match them with a glob
pattern. pattern.
Using the catalog for navigation reduces the overhead considerably because only Using the catalog for navigation reduces the overhead considerably because only
the catalog needs to be downloaded and, optionally, decrypted. the catalog needs to be downloaded and, optionally, decrypted.
The actual chunks are only accessed if the metadata in the catalog is The actual chunks are only accessed if the metadata in the catalog is not enough
insufficient or for the actual restore. or for the actual restore.
Similar to common UNIX shells, ``cd`` and ``ls`` are the commands used to change Similar to common UNIX shells ``cd`` and ``ls`` are the commands used to change
working directory and list directory contents in the archive. working directory and list directory contents in the archive.
``pwd`` shows the full path of the current working directory with respect to the ``pwd`` shows the full path of the current working directory with respect to the
archive root. archive root.
The ability to quickly search the contents of the archive is a commonly required Being able to quickly search the contents of the archive is a commonly needed feature.
feature. That's where the catalog is most valuable. For example: That's where the catalog is most valuable.
For example:
.. code-block:: console .. code-block:: console
@ -489,8 +455,8 @@ feature. That's where the catalog is most valuable. For example:
pxar:/ > restore-selected /target/path pxar:/ > restore-selected /target/path
... ...
This will find and print all files ending in ``.txt`` located in ``etc/`` or its This will find and print all files ending in ``.txt`` located in ``etc/`` or a
subdirectories, and add the corresponding pattern to the list for subsequent restores. subdirectory and add the corresponding pattern to the list for subsequent restores.
``list-selected`` shows these patterns and ``restore-selected`` finally restores ``list-selected`` shows these patterns and ``restore-selected`` finally restores
all files in the archive matching the patterns to ``/target/path`` on the local all files in the archive matching the patterns to ``/target/path`` on the local
host. This will scan the whole archive. host. This will scan the whole archive.
@ -515,7 +481,7 @@ Mounting of Archives via FUSE
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The :term:`FUSE` implementation for the pxar archive allows you to mount a The :term:`FUSE` implementation for the pxar archive allows you to mount a
file archive as a read-only filesystem to a mount point on your host. file archive as a read-only filesystem to a mountpoint on your host.
.. code-block:: console .. code-block:: console
@ -531,7 +497,7 @@ This allows you to access the full contents of the archive in a seamless manner.
load on your host, depending on the operations you perform on the mounted load on your host, depending on the operations you perform on the mounted
filesystem. filesystem.
To unmount the filesystem, use the ``umount`` command on the mount point: To unmount the filesystem use the ``umount`` command on the mountpoint:
.. code-block:: console .. code-block:: console
@ -540,7 +506,7 @@ To unmount the filesystem, use the ``umount`` command on the mount point:
Login and Logout Login and Logout
---------------- ----------------
The client tool prompts you to enter the login password as soon as you The client tool prompts you to enter the logon password as soon as you
want to access the backup server. The server checks your credentials want to access the backup server. The server checks your credentials
and responds with a ticket that is valid for two hours. The client and responds with a ticket that is valid for two hours. The client
tool automatically stores that ticket and uses it for further requests tool automatically stores that ticket and uses it for further requests
@ -569,7 +535,7 @@ Changing the Owner of a Backup Group
By default, the owner of a backup group is the user which was used to originally By default, the owner of a backup group is the user which was used to originally
create that backup group (or in the case of sync jobs, ``root@pam``). This create that backup group (or in the case of sync jobs, ``root@pam``). This
means that if a user ``mike@pbs`` created a backup, another user ``john@pbs`` means that if a user ``mike@pbs`` created a backup, another user ``john@pbs``
can not be used to create backups in that same backup group. In case you want can not be used to create backups in that same backup group. In case you want
to change the owner of a backup, you can do so with the below command, using a to change the owner of a backup, you can do so with the below command, using a
user that has ``Datastore.Modify`` privileges on the datastore. user that has ``Datastore.Modify`` privileges on the datastore.
@ -578,10 +544,10 @@ user that has ``Datastore.Modify`` privileges on the datastore.
# proxmox-backup-client change-owner vm/103 john@pbs # proxmox-backup-client change-owner vm/103 john@pbs
This can also be done from within the web interface, by navigating to the This can also be done from within the web interface, by navigating to the
`Content` section of the datastore that contains the backup group and selecting `Content` section of the datastore that contains the backup group and
the user icon under the `Actions` column. Common cases for this could be to selecting the user icon under the `Actions` column. Common cases for this could
change the owner of a sync job from ``root@pam``, or to repurpose a backup be to change the owner of a sync job from ``root@pam``, or to repurpose a
group. backup group.
.. _backup-pruning: .. _backup-pruning:
@ -589,24 +555,16 @@ group.
Pruning and Removing Backups Pruning and Removing Backups
---------------------------- ----------------------------
You can manually delete a backup snapshot using the ``forget`` command: You can manually delete a backup snapshot using the ``forget``
command:
.. code-block:: console .. code-block:: console
# proxmox-backup-client snapshot forget <snapshot> # proxmox-backup-client snapshot forget <snapshot>
.. caution:: This command removes all archives in this backup snapshot. They .. caution:: This command removes all archives in this backup
will be inaccessible and *unrecoverable*. snapshot. They will be inaccessible and unrecoverable.
Don't forget to add the namespace ``--ns`` parameter if you want to forget a
snapshot that is contained in the root namespace:
.. code-block:: console
# proxmox-backup-client snapshot forget <snapshot> --ns <ns>
Although manual removal is sometimes required, the ``prune`` Although manual removal is sometimes required, the ``prune``
@ -678,25 +636,6 @@ shows the list of existing snapshots and what actions prune would take.
in the chunk-store. The chunk-store still contains the data blocks. To free in the chunk-store. The chunk-store still contains the data blocks. To free
space you need to perform :ref:`client_garbage-collection`. space you need to perform :ref:`client_garbage-collection`.
It is also possible to protect single snapshots from being pruned or deleted:
.. code-block:: console
# proxmox-backup-client snapshot protected update <snapshot> true
This will set the protected flag on the snapshot and prevent pruning or manual
deletion of this snapshot untilt he flag is removed again with:
.. code-block:: console
# proxmox-backup-client snapshot protected update <snapshot> false
When a group is with a protected snapshot is deleted, only the non-protected
ones are removed and the group will remain.
.. note:: This flag will not be synced when using pull or sync jobs. If you
want to protect a synced snapshot, you have to manually to this again on
the target backup server.
.. _client_garbage-collection: .. _client_garbage-collection:
@ -722,7 +661,7 @@ unused data blocks are removed.
(access time) property. Filesystems are mounted with the ``relatime`` option (access time) property. Filesystems are mounted with the ``relatime`` option
by default. This results in a better performance by only updating the by default. This results in a better performance by only updating the
``atime`` property if the last access has been at least 24 hours ago. The ``atime`` property if the last access has been at least 24 hours ago. The
downside is that touching a chunk within these 24 hours will not always downside is, that touching a chunk within these 24 hours will not always
update its ``atime`` property. update its ``atime`` property.
Chunks in the grace period will be logged at the end of the garbage Chunks in the grace period will be logged at the end of the garbage
@ -746,8 +685,8 @@ unused data blocks are removed.
Average chunk size: 2486565 Average chunk size: 2486565
TASK OK TASK OK
Garbage collection can also be scheduled using ``promxox-backup-manager`` or
from the Proxmox Backup Server's web interface. .. todo:: howto run garbage-collection at regular intervals (cron)
Benchmarking Benchmarking
------------ ------------

View File

@ -1,10 +1,10 @@
Backup Protocol Backup Protocol
=============== ===============
Proxmox Backup Server uses a REST-based API. While the management Proxmox Backup Server uses a REST based API. While the management
interface uses normal HTTP, the actual backup and restore interface uses interface use normal HTTP, the actual backup and restore interface use
HTTP/2 for improved performance. Both HTTP and HTTP/2 are well known HTTP/2 for improved performance. Both HTTP and HTTP/2 are well known
standards, so the following section assumes that you are familiar with standards, so the following section assumes that you are familiar on
how to use them. how to use them.
@ -13,35 +13,35 @@ Backup Protocol API
To start a new backup, the API call ``GET /api2/json/backup`` needs to To start a new backup, the API call ``GET /api2/json/backup`` needs to
be upgraded to a HTTP/2 connection using be upgraded to a HTTP/2 connection using
``proxmox-backup-protocol-v1`` as the protocol name:: ``proxmox-backup-protocol-v1`` as protocol name::
GET /api2/json/backup HTTP/1.1 GET /api2/json/backup HTTP/1.1
UPGRADE: proxmox-backup-protocol-v1 UPGRADE: proxmox-backup-protocol-v1
The server replies with the ``HTTP 101 Switching Protocol`` status code, The server replies with HTTP 101 Switching Protocol status code,
and you can then issue REST commands on the updated HTTP/2 connection. and you can then issue REST commands on that updated HTTP/2 connection.
The backup protocol allows you to upload three different kind of files: The backup protocol allows you to upload three different kind of files:
- Chunks and blobs (binary data) - Chunks and blobs (binary data)
- Fixed indexes (List of chunks with fixed size) - Fixed Indexes (List of chunks with fixed size)
- Dynamic indexes (List of chunks with variable size) - Dynamic Indexes (List of chunk with variable size)
The following section provides a short introduction on how to upload such The following section gives a short introduction how to upload such
files. Please use the `API Viewer <api-viewer/index.html>`_ for files. Please use the `API Viewer <api-viewer/index.html>`_ for
details about the available REST commands. details about available REST commands.
Upload Blobs Upload Blobs
~~~~~~~~~~~~ ~~~~~~~~~~~~
Blobs are uploaded using ``POST /blob``. The HTTP body contains the Uploading blobs is done using ``POST /blob``. The HTTP body contains the
data encoded as :ref:`Data Blob <data-blob-format>`. data encoded as :ref:`Data Blob <data-blob-format>`).
The file name must end with ``.blob``, and is automatically added The file name needs to end with ``.blob``, and is automatically added
to the backup manifest, following the call to ``POST /finish``. to the backup manifest.
Upload Chunks Upload Chunks
@ -56,41 +56,40 @@ encoded as :ref:`Data Blob <data-blob-format>`).
Upload Fixed Indexes Upload Fixed Indexes
~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~
Fixed indexes are used to store VM image data. The VM image is split Fixed indexes are use to store VM image data. The VM image is split
into equally sized chunks, which are uploaded individually. The index into equally sized chunks, which are uploaded individually. The index
file simply contains a list of chunk digests. file simply contains a list to chunk digests.
You create a fixed index with ``POST /fixed_index``. Then, upload You create a fixed index with ``POST /fixed_index``. Then upload
chunks with ``POST /fixed_chunk``, and append them to the index with chunks with ``POST /fixed_chunk``, and append them to the index with
``PUT /fixed_index``. When finished, you need to close the index using ``PUT /fixed_index``. When finished, you need to close the index using
``POST /fixed_close``. ``POST /fixed_close``.
The file name needs to end with ``.fidx``, and is automatically added The file name needs to end with ``.fidx``, and is automatically added
to the backup manifest, following the call to ``POST /finish``. to the backup manifest.
Upload Dynamic Indexes Upload Dynamic Indexes
~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~
Dynamic indexes are used to store file archive data. The archive data Dynamic indexes are use to store file archive data. The archive data
is split into dynamically sized chunks, which are uploaded is split into dynamically sized chunks, which are uploaded
individually. The index file simply contains a list of chunk digests individually. The index file simply contains a list to chunk digests
and offsets. and offsets.
You can create a dynamically sized index with ``POST /dynamic_index``. Then, You create a dynamic sized index with ``POST /dynamic_index``. Then
upload chunks with ``POST /dynamic_chunk``, and append them to the index with upload chunks with ``POST /dynamic_chunk``, and append them to the index with
``PUT /dynamic_index``. When finished, you need to close the index using ``PUT /dynamic_index``. When finished, you need to close the index using
``POST /dynamic_close``. ``POST /dynamic_close``.
The filename needs to end with ``.didx``, and is automatically added The file name needs to end with ``.didx``, and is automatically added
to the backup manifest, following the call to ``POST /finish``. to the backup manifest.
Finish Backup Finish Backup
~~~~~~~~~~~~~ ~~~~~~~~~~~~~
Once you have uploaded all data, you need to call ``POST /finish``. This Once you have uploaded all data, you need to call ``POST
commits all data and ends the backup protocol. /finish``. This commits all data and ends the backup protocol.
Restore/Reader Protocol API Restore/Reader Protocol API
@ -103,39 +102,39 @@ be upgraded to a HTTP/2 connection using
GET /api2/json/reader HTTP/1.1 GET /api2/json/reader HTTP/1.1
UPGRADE: proxmox-backup-reader-protocol-v1 UPGRADE: proxmox-backup-reader-protocol-v1
The server replies with the ``HTTP 101 Switching Protocol`` status code, The server replies with HTTP 101 Switching Protocol status code,
and you can then issue REST commands on that updated HTTP/2 connection. and you can then issue REST commands on that updated HTTP/2 connection.
The reader protocol allows you to download three different kinds of files: The reader protocol allows you to download three different kind of files:
- Chunks and blobs (binary data) - Chunks and blobs (binary data)
- Fixed indexes (list of chunks with fixed size) - Fixed Indexes (List of chunks with fixed size)
- Dynamic indexes (list of chunks with variable size) - Dynamic Indexes (List of chunk with variable size)
The following section provides a short introduction on how to download such The following section gives a short introduction how to download such
files. Please use the `API Viewer <api-viewer/index.html>`_ for details about files. Please use the `API Viewer <api-viewer/index.html>`_ for details about
the available REST commands. available REST commands.
Download Blobs Download Blobs
~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~
Blobs are downloaded using ``GET /download``. The HTTP body contains the Downloading blobs is done using ``GET /download``. The HTTP body contains the
data encoded as :ref:`Data Blob <data-blob-format>`. data encoded as :ref:`Data Blob <data-blob-format>`.
Download Chunks Download Chunks
~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~
Chunks are downloaded using ``GET /chunk``. The HTTP body contains the Downloading chunks is done using ``GET /chunk``. The HTTP body contains the
data encoded as :ref:`Data Blob <data-blob-format>`. data encoded as :ref:`Data Blob <data-blob-format>`).
Download Index Files Download Index Files
~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~
Index files are downloaded using ``GET /download``. The HTTP body Downloading index files is done using ``GET /download``. The HTTP body
contains the data encoded as :ref:`Fixed Index <fixed-index-format>` contains the data encoded as :ref:`Fixed Index <fixed-index-format>`
or :ref:`Dynamic Index <dynamic-index-format>`. or :ref:`Dynamic Index <dynamic-index-format>`.

View File

@ -37,7 +37,7 @@ Each field can contain multiple values in the following formats:
* and a combination of the above: e.g., 01,05..10,12/02 * and a combination of the above: e.g., 01,05..10,12/02
* or a `*` for every possible value: e.g., \*:00 * or a `*` for every possible value: e.g., \*:00
There are some special values that have a specific meaning: There are some special values that have specific meaning:
================================= ============================== ================================= ==============================
Value Syntax Value Syntax
@ -81,19 +81,19 @@ Not all features of systemd calendar events are implemented:
* no Unix timestamps (e.g. `@12345`): instead use date and time to specify * no Unix timestamps (e.g. `@12345`): instead use date and time to specify
a specific point in time a specific point in time
* no timezone: all schedules use the timezone of the server * no timezone: all schedules use the set timezone on the server
* no sub-second resolution * no sub-second resolution
* no reverse day syntax (e.g. 2020-03~01) * no reverse day syntax (e.g. 2020-03~01)
* no repetition of ranges (e.g. 1..10/2) * no repetition of ranges (e.g. 1..10/2)
Notes on Scheduling Notes on scheduling
------------------- -------------------
In `Proxmox Backup`_, scheduling for most tasks is done in the In `Proxmox Backup`_ scheduling for most tasks is done in the
`proxmox-backup-proxy`. This daemon checks all job schedules `proxmox-backup-proxy`. This daemon checks all job schedules
every minute, to see if any are due. This means that even though if they are due every minute. This means that even if
`calendar events` can contain seconds, it will only be checked `calendar events` can contain seconds, it will only be checked
once per minute. once a minute.
Also, all schedules will be checked against the timezone set Also, all schedules will be checked against the timezone set
in the `Proxmox Backup`_ server. in the `Proxmox Backup`_ server.

View File

@ -1,333 +0,0 @@
.. _sysadmin_certificate_management:
Certificate Management
----------------------
Access to the API and thus the web-based administration interface is always
encrypted through ``https``. Each `Proxmox Backup`_ host creates by default its
own (self-signed) certificate. This certificate is used for encrypted
communication with the hosts ``proxmox-backup-proxy`` service, for any API
call between a user or backup-client and the web-interface.
Certificate verification when sending backups to a `Proxmox Backup`_ server
is either done based on pinning the certificate fingerprints in the storage/remote
configuration, or by using certificates, signed by a trusted certificate authority.
.. _sysadmin_certs_api_gui:
Certificates for the API and SMTP
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
`Proxmox Backup`_ stores it certificate and key in:
- ``/etc/proxmox-backup/proxy.pem``
- ``/etc/proxmox-backup/proxy.key``
You have the following options for the certificate:
1. Keep using the default self-signed certificate in
``/etc/proxmox-backup/proxy.pem``.
2. Use an externally provided certificate (for example, signed by a
commercial Certificate Authority (CA)).
3. Use an ACME provider like Lets Encrypt to get a trusted certificate
with automatic renewal; this is also integrated in the `Proxmox Backup`_
API and web interface.
Certificates are managed through the `Proxmox Backup`_
web-interface/API or using the the ``proxmox-backup-manager`` CLI tool.
.. _sysadmin_certs_upload_custom:
Upload Custom Certificate
~~~~~~~~~~~~~~~~~~~~~~~~~
If you already have a certificate which you want to use for a Proxmox
Mail Gateway host, you can simply upload that certificate over the web
interface.
.. image:: images/screenshots/pbs-gui-certs-upload-custom.png
:align: right
:alt: Upload a custom certificate
Note that any certificate key files must not be password protected.
.. _sysadmin_certs_get_trusted_acme_cert:
Trusted certificates via Lets Encrypt (ACME)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
`Proxmox Backup`_ includes an implementation of the **A**\ utomatic
**C**\ ertificate **M**\ anagement **E**\ nvironment (**ACME**)
protocol, allowing `Proxmox Backup`_ admins to use an ACME provider
like Lets Encrypt for easy setup of TLS certificates, which are
accepted and trusted by modern operating systems and web browsers out of
the box.
Currently, the two ACME endpoints implemented are the `Lets Encrypt
(LE) <https://letsencrypt.org>`_ production and staging environments.
Our ACME client supports validation of ``http-01`` challenges using a
built-in web server and validation of ``dns-01`` challenges using a DNS
plugin supporting all the DNS API endpoints
`acme.sh <https://acme.sh>`_ does.
.. _sysadmin_certs_acme_account:
ACME Account
^^^^^^^^^^^^
.. image:: images/screenshots/pbs-gui-acme-create-account.png
:align: right
:alt: Create ACME Account
You need to register an ACME account per cluster, with the endpoint you
want to use. The email address used for that account will serve as the
contact point for renewal-due or similar notifications from the ACME
endpoint.
You can register or deactivate ACME accounts over the web interface
``Certificates -> ACME Accounts`` or using the ``proxmox-backup-manager`` command
line tool.
::
proxmox-backup-manager acme account register <account-name> <mail@example.com>
.. tip::
Because of
`rate-limits <https://letsencrypt.org/docs/rate-limits/>`_ you
should use LE ``staging`` for experiments or if you use ACME for the
very first time until all is working there, and only then switch over
to the production directory.
.. _sysadmin_certs_acme_plugins:
ACME Plugins
^^^^^^^^^^^^
The ACME plugins role is to provide automatic verification that you,
and thus the `Proxmox Backup`_ server under your operation, are the
real owner of a domain. This is the basic building block of automatic
certificate management.
The ACME protocol specifies different types of challenges, for example
the ``http-01``, where a web server provides a file with a specific
token to prove that it controls a domain. Sometimes this isnt possible,
either because of technical limitations or if the address of a record is
not reachable from the public internet. The ``dns-01`` challenge can be
used in such cases. This challenge is fulfilled by creating a certain
DNS record in the domains zone.
.. image:: images/screenshots/pbs-gui-acme-create-challenge-plugin.png
:align: right
:alt: Create ACME Account
`Proxmox Backup`_ supports both of those challenge types out of the
box, you can configure plugins either over the web interface under
``Certificates -> ACME Challenges``, or using the
``proxmox-backup-manager acme plugin add`` command.
ACME Plugin configurations are stored in ``/etc/proxmox-backup/acme/plugins.cfg``.
.. _domains:
Domains
^^^^^^^
You can add new or manage existing domain entries under
``Certificates``, or using the ``proxmox-backup-manager`` command.
.. image:: images/screenshots/pbs-gui-acme-add-domain.png
:align: right
:alt: Add a Domain for ACME verification
After configuring the desired domain(s) for a node and ensuring that the
desired ACME account is selected, you can order your new certificate
over the web-interface. On success, the interface will reload after
roughly 10 seconds.
Renewal will happen `automatically <#sysadmin-certs-acme-automatic-renewal>`_
.. _sysadmin_certs_acme_http_challenge:
ACME HTTP Challenge Plugin
~~~~~~~~~~~~~~~~~~~~~~~~~~
There is always an implicitly configured ``standalone`` plugin for
validating ``http-01`` challenges via the built-in web server spawned on
port 80.
.. note::
The name ``standalone`` means that it can provide the validation on
its own, without any third party service.
There are a few prerequisites to use this for certificate management
with Lets Encrypts ACME.
- You have to accept the ToS of Lets Encrypt to register an account.
- **Port 80** of the node needs to be reachable from the internet.
- There **must** be no other listener on port 80.
- The requested (sub)domain needs to resolve to a public IP of the
`Proxmox Backup`_ host.
.. _sysadmin_certs_acme_dns_challenge:
ACME DNS API Challenge Plugin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On systems where external access for validation via the ``http-01``
method is not possible or desired, it is possible to use the ``dns-01``
validation method. This validation method requires a DNS server that
allows provisioning of ``TXT`` records via an API.
.. _sysadmin_certs_acme_dns_api_config:
Configuring ACME DNS APIs for validation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
`Proxmox Backup`_ re-uses the DNS plugins developed for the
``acme.sh`` [1]_ project. Please refer to its documentation for details
on configuration of specific APIs.
The easiest way to configure a new plugin with the DNS API is using the
web interface (``Certificates -> ACME Accounts/Challenges``).
Here you can add a new challenge plugin by selecting your API provider
and entering the credential data to access your account over their API.
.. tip::
See the acme.sh `How to use DNS
API <https://github.com/acmesh-official/acme.sh/wiki/dnsapi#how-to-use-dns-api>`_
wiki for more detailed information about getting API credentials for
your provider. Configuration values do not need to be quoted with
single or double quotes; for some plugins that is even an error.
As there are many DNS providers and API endpoints, `Proxmox Backup`_
automatically generates the form for the credentials, but not all
providers are annotated yet. For those you will see a bigger text area,
into which you simply need to copy all the credentials
``KEY``\ =\ ``VALUE`` pairs.
.. _dns_validation_through_cname_alias:
DNS Validation through CNAME Alias
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A special ``alias`` mode can be used to handle validation on a different
domain/DNS server, in case your primary/real DNS does not support
provisioning via an API. Manually set up a permanent ``CNAME`` record
for ``_acme-challenge.domain1.example`` pointing to
``_acme-challenge.domain2.example``, and set the ``alias`` property in
the `Proxmox Backup`_ node configuration file ``/etc/proxmox-backup/node.cfg``
to ``domain2.example`` to allow the DNS server of ``domain2.example`` to
validate all challenges for ``domain1.example``.
.. _sysadmin_certs_acme_dns_wildcard:
Wildcard Certificates
^^^^^^^^^^^^^^^^^^^^^
Wildcard DNS names start with a ``*.`` prefix and are considered valid
for all (one-level) subdomain names of the verified domain. So a
certificate for ``*.domain.example`` is valid for ``foo.domain.example``
and ``bar.domain.example``, but not for ``baz.foo.domain.example``.
Currently, you can only create wildcard certificates with the `DNS
challenge
type <https://letsencrypt.org/docs/challenge-types/#dns-01-challenge>`_.
.. _combination_of_plugins:
Combination of Plugins
^^^^^^^^^^^^^^^^^^^^^^
Combining ``http-01`` and ``dns-01`` validation is possible in case your
node is reachable via multiple domains with different requirements / DNS
provisioning capabilities. Mixing DNS APIs from multiple providers or
instances is also possible by specifying different plugin instances per
domain.
.. tip::
Accessing the same service over multiple domains increases complexity
and should be avoided if possible.
.. _sysadmin_certs_acme_automatic_renewal:
Automatic renewal of ACME certificates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If a node has been successfully configured with an ACME-provided
certificate (either via ``proxmox-backup-manager`` or via the web-interface/API), the
certificate will be renewed automatically by the ``proxmox-backup-daily-update.service``.
Currently, renewal is triggered if the certificate either has already
expired or if it will expire in the next 30 days.
.. _manually_change_certificate_over_command_line:
Manually Change Certificate over Command-Line
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you want to get rid of certificate verification warnings, you have to
generate a valid certificate for your server.
Log in to your `Proxmox Backup`_ via ssh or use the console:
::
openssl req -newkey rsa:2048 -nodes -keyout key.pem -out req.pem
Follow the instructions on the screen, for example:
::
Country Name (2 letter code) [AU]: AT
State or Province Name (full name) [Some-State]:Vienna
Locality Name (eg, city) []:Vienna
Organization Name (eg, company) [Internet Widgits Pty Ltd]: Proxmox GmbH
Organizational Unit Name (eg, section) []:Proxmox Backup
Common Name (eg, YOUR name) []: yourproxmox.yourdomain.com
Email Address []:support@yourdomain.com
Please enter the following 'extra' attributes to be sent with your certificate request
A challenge password []: not necessary
An optional company name []: not necessary
After you have finished the certificate request, you have to send the
file ``req.pem`` to your Certification Authority (CA). The CA will issue
the certificate (BASE64 encoded), based on your request save this file
as ``cert.pem`` to your `Proxmox Backup`_.
To activate the new certificate, do the following on your `Proxmox Backup`_
::
cp key.pem /etc/proxmox-backup/proxy.key
cp cert.pem /etc/proxmox-backup/proxy.pem
Then restart the API servers:
::
systemctl restart proxmox-backup-proxy
Test your new certificate, using your browser.
.. note::
To transfer files to and from your `Proxmox Backup`_, you can use
secure copy: If your desktop runs Linux, you can use the ``scp``
command line tool. If your desktop PC runs windows, please use an scp
client like WinSCP (see https://winscp.net/).
.. [1]
acme.sh https://github.com/acmesh-official/acme.sh

View File

@ -6,37 +6,18 @@ Command Line Tools
.. include:: proxmox-backup-client/description.rst .. include:: proxmox-backup-client/description.rst
``proxmox-file-restore``
~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: proxmox-file-restore/description.rst
``proxmox-backup-manager`` ``proxmox-backup-manager``
~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: proxmox-backup-manager/description.rst .. include:: proxmox-backup-manager/description.rst
``proxmox-tape``
~~~~~~~~~~~~~~~~
.. include:: proxmox-tape/description.rst
``pmt``
~~~~~~~
.. include:: pmt/description.rst
``pmtx``
~~~~~~~~
.. include:: pmtx/description.rst
``pxar`` ``pxar``
~~~~~~~~ ~~~~~~~~
.. include:: pxar/description.rst .. include:: pxar/description.rst
``proxmox-file-restore``
~~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: proxmox-file-restore/description.rst
``proxmox-backup-debug``
~~~~~~~~~~~~~~~~~~~~~~~~
.. include:: proxmox-backup-debug/description.rst

View File

@ -10,7 +10,7 @@ Command Syntax
Catalog Shell Commands Catalog Shell Commands
~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~
The following commands are available in an interactive restore shell: Those command are available when you start an interactive restore shell:
.. code-block:: console .. code-block:: console
@ -51,13 +51,3 @@ The following commands are available in an interactive restore shell:
-------- --------
.. include:: pxar/synopsis.rst .. include:: pxar/synopsis.rst
``proxmox-file-restore``
------------------------
.. include:: proxmox-file-restore/synopsis.rst
``proxmox-backup-debug``
------------------------
.. include:: proxmox-backup-debug/synopsis.rst

View File

@ -77,7 +77,7 @@ project = 'Proxmox Backup'
copyright = '2019-2021, Proxmox Server Solutions GmbH' copyright = '2019-2021, Proxmox Server Solutions GmbH'
author = 'Proxmox Support Team' author = 'Proxmox Support Team'
# The version info for the project you're documenting, acts as a replacement for # The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the # |version| and |release|, also used in various other places throughout the
# built documents. # built documents.
# #
@ -108,14 +108,11 @@ today_fmt = '%A, %d %B %Y'
exclude_patterns = [ exclude_patterns = [
'_build', 'Thumbs.db', '.DS_Store', '_build', 'Thumbs.db', '.DS_Store',
'*/man1.rst', '*/man1.rst',
'certificate-management.rst',
'config/*/man5.rst', 'config/*/man5.rst',
'epilog.rst', 'epilog.rst',
'pbs-copyright.rst', 'pbs-copyright.rst',
'local-zfs.rst', 'local-zfs.rst'
'package-repositories.rst', 'package-repositories.rst',
'system-booting.rst',
'traffic-control.rst',
] ]
# The reST default role (used for this markup: `text`) to use for all # The reST default role (used for this markup: `text`) to use for all

View File

@ -2,13 +2,13 @@ This file contains the access control list for the Proxmox Backup
Server API. Server API.
Each line starts with ``acl:``, followed by 4 additional values Each line starts with ``acl:``, followed by 4 additional values
separated by colon. separated by collon.
:propagate: Propagate permissions down the hierarchy :propagate: Propagate permissions down the hierachrchy
:path: The object path :path: The object path
:User/Token: List of users and tokens :User/Token: List of users and token
:Role: List of assigned roles :Role: List of assigned roles

View File

@ -1,5 +1,5 @@
This file contains a list of datastore configuration sections. Each The file contains a list of datastore configuration sections. Each
section starts with the header ``datastore: <name>``, followed by the section starts with a header ``datastore: <name>``, followed by the
datastore configuration options. datastore configuration options.
:: ::

View File

@ -1,4 +1,4 @@
Each entry starts with the header ``pool: <name>``, followed by the Each entry starts with a header ``pool: <name>``, followed by the
media pool configuration options. media pool configuration options.
:: ::

View File

@ -1,6 +1,6 @@
This file contains information used to access remote servers. This file contains information used to access remote servers.
Each entry starts with the header ``remote: <name>``, followed by the Each entry starts with a header ``remote: <name>``, followed by the
remote configuration options. remote configuration options.
:: ::

View File

@ -1,4 +1,4 @@
Each entry starts with the header ``sync: <name>``, followed by the Each entry starts with a header ``sync: <name>``, followed by the
job configuration options. job configuration options.
:: ::

View File

@ -1,4 +1,4 @@
Each entry starts with the header ``backup: <name>``, followed by the Each entry starts with a header ``backup: <name>``, followed by the
job configuration options. job configuration options.
:: ::

View File

@ -1,7 +1,7 @@
Each LTO drive configuration section starts with the header ``lto: <name>``, Each LTO drive configuration section starts with a header ``lto: <name>``,
followed by the drive configuration options. followed by the drive configuration options.
Tape changer configurations start with the header ``changer: <name>``, Tape changer configurations starts with ``changer: <name>``,
followed by the changer configuration options. followed by the changer configuration options.
:: ::
@ -18,5 +18,5 @@ followed by the changer configuration options.
You can use the ``proxmox-tape drive`` and ``proxmox-tape changer`` You can use the ``proxmox-tape drive`` and ``proxmox-tape changer``
commands to manipulate this file. commands to manipulate this file.
.. NOTE:: The ``virtual:`` drive type is experimental and should only be used .. NOTE:: The ``virtual:`` drive type is experimental and onyl used
for debugging. for debugging.

View File

@ -1,9 +1,9 @@
This file contains the list of API users and API tokens. This file contains the list of API users and API tokens.
Each user configuration section starts with the header ``user: <name>``, Each user configuration section starts with a header ``user: <name>``,
followed by the user configuration options. followed by the user configuration options.
API token configuration starts with the header ``token: API token configuration starts with a header ``token:
<userid!token_name>``, followed by the token configuration. The data <userid!token_name>``, followed by the token configuration. The data
used to authenticate tokens is stored in a separate file used to authenticate tokens is stored in a separate file
(``token.shadow``). (``token.shadow``).

View File

@ -1,4 +1,4 @@
Each entry starts with the header ``verification: <name>``, followed by the Each entry starts with a header ``verification: <name>``, followed by the
job configuration options. job configuration options.
:: ::

View File

@ -1,7 +1,7 @@
Configuration Files Configuration Files
=================== ===================
All Proxmox Backup Server configuration files reside in the directory All Proxmox Backup Server configuration files resides inside directory
``/etc/proxmox-backup/``. ``/etc/proxmox-backup/``.

View File

@ -13,6 +13,7 @@
.. _Proxmox: https://www.proxmox.com .. _Proxmox: https://www.proxmox.com
.. _Proxmox Community Forum: https://forum.proxmox.com .. _Proxmox Community Forum: https://forum.proxmox.com
.. _Proxmox Virtual Environment: https://www.proxmox.com/proxmox-ve .. _Proxmox Virtual Environment: https://www.proxmox.com/proxmox-ve
.. FIXME
.. _Proxmox Backup: https://pbs.proxmox.com/wiki/index.php/Main_Page .. _Proxmox Backup: https://pbs.proxmox.com/wiki/index.php/Main_Page
.. _PBS Development List: https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel .. _PBS Development List: https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
.. _reStructuredText: https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html .. _reStructuredText: https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html
@ -22,7 +23,6 @@
.. _Virtual machine: https://en.wikipedia.org/wiki/Virtual_machine .. _Virtual machine: https://en.wikipedia.org/wiki/Virtual_machine
.. _APT: http://en.wikipedia.org/wiki/Advanced_Packaging_Tool .. _APT: http://en.wikipedia.org/wiki/Advanced_Packaging_Tool
.. _QEMU: https://www.qemu.org/ .. _QEMU: https://www.qemu.org/
.. _LXC: https://linuxcontainers.org/lxc/introduction/
.. _Client-server model: https://en.wikipedia.org/wiki/Client-server_model .. _Client-server model: https://en.wikipedia.org/wiki/Client-server_model
.. _AE: https://en.wikipedia.org/wiki/Authenticated_encryption .. _AE: https://en.wikipedia.org/wiki/Authenticated_encryption
@ -35,7 +35,7 @@
.. _ZFS: https://en.wikipedia.org/wiki/ZFS .. _ZFS: https://en.wikipedia.org/wiki/ZFS
.. _Proxmox VE: https://pve.proxmox.com .. _Proxmox VE: https://pve.proxmox.com
.. _RFC3339: https://tools.ietf.org/html/rfc3339 .. _RFC3399: https://tools.ietf.org/html/rfc3339
.. _UTC: https://en.wikipedia.org/wiki/Coordinated_Universal_Time .. _UTC: https://en.wikipedia.org/wiki/Coordinated_Universal_Time
.. _ISO Week date: https://en.wikipedia.org/wiki/ISO_week_date .. _ISO Week date: https://en.wikipedia.org/wiki/ISO_week_date

View File

@ -24,13 +24,11 @@ future plans to support 32-bit processors.
How long will my Proxmox Backup Server version be supported? How long will my Proxmox Backup Server version be supported?
------------------------------------------------------------ ------------------------------------------------------------
+-----------------------+----------------------+---------------+------------+--------------------+ +-----------------------+--------------------+---------------+------------+--------------------+
|Proxmox Backup Version | Debian Version | First Release | Debian EOL | Proxmox Backup EOL | |Proxmox Backup Version | Debian Version | First Release | Debian EOL | Proxmox Backup EOL |
+=======================+======================+===============+============+====================+ +=======================+====================+===============+============+====================+
|Proxmox Backup 2.x | Debian 11 (Bullseye) | 2021-07 | tba | tba | |Proxmox Backup 1.x | Debian 10 (Buster) | 2020-11 | tba | tba |
+-----------------------+----------------------+---------------+------------+--------------------+ +-----------------------+--------------------+---------------+------------+--------------------+
|Proxmox Backup 1.x | Debian 10 (Buster) | 2020-11 | 2022-08 | 2022-07 |
+-----------------------+----------------------+---------------+------------+--------------------+
Can I copy or synchronize my datastore to another location? Can I copy or synchronize my datastore to another location?
@ -69,6 +67,6 @@ be able to read the data.
Is the backup incremental/deduplicated? Is the backup incremental/deduplicated?
--------------------------------------- ---------------------------------------
With Proxmox Backup Server, backups are sent incrementally to the server, and With Proxmox Backup Server, backups are sent incremental and data is
data is then deduplicated on the server. This minimizes both the storage deduplicated on the server.
consumed and the impact on the network. This minimizes both the storage consumed and the network impact.

View File

@ -14,8 +14,7 @@ Proxmox File Archive Format (``.pxar``)
Data Blob Format (``.blob``) Data Blob Format (``.blob``)
---------------------------- ----------------------------
The data blob format is used to store small binary data. The magic number The data blob format is used to store small binary data. The magic number decides the exact format:
decides the exact format:
.. list-table:: .. list-table::
:widths: auto :widths: auto
@ -33,8 +32,7 @@ decides the exact format:
- encrypted - encrypted
- compressed - compressed
The compression algorithm used is ``zstd``. The encryption cipher is Compression algorithm is ``zstd``. Encryption cipher is ``AES_256_GCM``.
``AES_256_GCM``.
Unencrypted blobs use the following format: Unencrypted blobs use the following format:
@ -45,15 +43,15 @@ Unencrypted blobs use the following format:
* - ``CRC32: [u8; 4]`` * - ``CRC32: [u8; 4]``
* - ``Data: (max 16MiB)`` * - ``Data: (max 16MiB)``
Encrypted blobs additionally contain a 16 byte initialization vector (IV), Encrypted blobs additionally contains a 16 byte IV, followed by a 16
followed by a 16 byte authenticated encryption (AE) tag, followed by the byte Authenticated Encyryption (AE) tag, followed by the encrypted
encrypted data: data:
.. list-table:: .. list-table::
* - ``MAGIC: [u8; 8]`` * - ``MAGIC: [u8; 8]``
* - ``CRC32: [u8; 4]`` * - ``CRC32: [u8; 4]``
* - ``IV: [u8; 16]`` * - ``ÌV: [u8; 16]``
* - ``TAG: [u8; 16]`` * - ``TAG: [u8; 16]``
* - ``Data: (max 16MiB)`` * - ``Data: (max 16MiB)``
@ -74,19 +72,19 @@ All numbers are stored as little-endian.
* - ``ctime: i64``, * - ``ctime: i64``,
- Creation Time (epoch) - Creation Time (epoch)
* - ``index_csum: [u8; 32]``, * - ``index_csum: [u8; 32]``,
- SHA-256 over the index (without header) ``SHA256(digest1||digest2||...)`` - Sha256 over the index (without header) ``SHA256(digest1||digest2||...)``
* - ``size: u64``, * - ``size: u64``,
- Image size - Image size
* - ``chunk_size: u64``, * - ``chunk_size: u64``,
- Chunk size - Chunk size
* - ``reserved: [u8; 4016]``, * - ``reserved: [u8; 4016]``,
- Overall header size is one page (4096 bytes) - overall header size is one page (4096 bytes)
* - ``digest1: [u8; 32]`` * - ``digest1: [u8; 32]``
- First chunk digest - first chunk digest
* - ``digest2: [u8; 32]`` * - ``digest2: [u8; 32]``
- Second chunk digest - next chunk
* - ... * - ...
- Next chunk digest ... - next chunk ...
.. _dynamic-index-format: .. _dynamic-index-format:
@ -105,16 +103,16 @@ All numbers are stored as little-endian.
* - ``ctime: i64``, * - ``ctime: i64``,
- Creation Time (epoch) - Creation Time (epoch)
* - ``index_csum: [u8; 32]``, * - ``index_csum: [u8; 32]``,
- SHA-256 over the index (without header) ``SHA256(offset1||digest1||offset2||digest2||...)`` - Sha256 over the index (without header) ``SHA256(offset1||digest1||offset2||digest2||...)``
* - ``reserved: [u8; 4032]``, * - ``reserved: [u8; 4032]``,
- Overall header size is one page (4096 bytes) - Overall header size is one page (4096 bytes)
* - ``offset1: u64`` * - ``offset1: u64``
- End of first chunk - End of first chunk
* - ``digest1: [u8; 32]`` * - ``digest1: [u8; 32]``
- First chunk digest - first chunk digest
* - ``offset2: u64`` * - ``offset2: u64``
- End of second chunk - End of second chunk
* - ``digest2: [u8; 32]`` * - ``digest2: [u8; 32]``
- Second chunk digest - second chunk digest
* - ... * - ...
- Next chunk offset/digest - next chunk offset/digest

View File

@ -11,7 +11,7 @@ Glossary
`Container`_ `Container`_
A container is an isolated user space. Programs run directly on A container is an isolated user space. Programs run directly on
the host's kernel, but with limited access to the host's resources. the host's kernel, but with limited access to the host resources.
Datastore Datastore
@ -23,19 +23,19 @@ Glossary
Rust is a new, fast and memory-efficient system programming Rust is a new, fast and memory-efficient system programming
language. It has no runtime or garbage collector. Rusts rich type language. It has no runtime or garbage collector. Rusts rich type
system and ownership model guarantee memory-safety and system and ownership model guarantee memory-safety and
thread-safety. This can eliminate many classes of bugs thread-safety. I can eliminate many classes of bugs
at compile-time. at compile-time.
`Sphinx`_ `Sphinx`_
Is a tool that makes it easy to create intelligent and nicely formatted Is a tool that makes it easy to create intelligent and
documentation. It was originally created for the documentation of the beautiful documentation. It was originally created for the
Python programming language. It has excellent facilities for the documentation of the Python programming language. It has excellent facilities for the
documentation of software projects in a range of languages. documentation of software projects in a range of languages.
`reStructuredText`_ `reStructuredText`_
Is an easy-to-read, what-you-see-is-what-you-get, plaintext Is an easy-to-read, what-you-see-is-what-you-get plaintext
markup syntax and parser system. markup syntax and parser system.
`FUSE` `FUSE`

View File

@ -8,9 +8,8 @@ tools. The web interface also provides a built-in console, so if you prefer the
command line or need some extra control, you have this option. command line or need some extra control, you have this option.
The web interface can be accessed via https://youripaddress:8007. The default The web interface can be accessed via https://youripaddress:8007. The default
login is `root`, and the password is either the one specified during the login is `root`, and the password is the one specified during the installation
installation process or the password of the root user, in case of installation process.
on top of Debian.
Features Features
@ -49,13 +48,12 @@ GUI Overview
The Proxmox Backup Server web interface consists of 3 main sections: The Proxmox Backup Server web interface consists of 3 main sections:
* **Header**: At the top. This shows version information and contains buttons to * **Header**: At the top. This shows version information, and contains buttons to view
view documentation, monitor running tasks, set the language, configure various documentation, monitor running tasks, set the language and logout.
display settings, and logout. * **Sidebar**: On the left. This contains the configuration options for
* **Sidebar**: On the left. This contains the administration options for
the server. the server.
* **Configuration Panel**: In the center. This contains the respective control * **Configuration Panel**: In the center. This contains the control interface for the
interfaces for the administration options in the *Sidebar*. configuration options in the *Sidebar*.
Sidebar Sidebar
@ -76,14 +74,12 @@ previous and currently running tasks, and subscription information.
Configuration Configuration
^^^^^^^^^^^^^ ^^^^^^^^^^^^^
The Configuration section contains some system options, such as time, network, The Configuration section contains some system configuration options, such as
WebAuthn, and HTTP proxy configuration. It also contains the following time and network configuration. It also contains the following subsections:
subsections:
* **Access Control**: Add and manage users, API tokens, and the permissions * **Access Control**: Add and manage users, API tokens, and the permissions
associated with these items associated with these items
* **Remotes**: Add, edit and remove remotes (see :term:`Remote`) * **Remotes**: Add, edit and remove remotes (see :term:`Remote`)
* **Certificates**: Manage ACME accounts and create SSL certificates.
* **Subscription**: Upload a subscription key, view subscription status and * **Subscription**: Upload a subscription key, view subscription status and
access a text-based system report. access a text-based system report.
@ -102,7 +98,6 @@ tasks and information. These are:
resource usage statistics resource usage statistics
* **Services**: Manage and monitor system services * **Services**: Manage and monitor system services
* **Updates**: An interface for upgrading packages * **Updates**: An interface for upgrading packages
* **Repositories**: An interface for configuring APT repositories
* **Syslog**: View log messages from the server * **Syslog**: View log messages from the server
* **Tasks**: Task history with multiple filter options * **Tasks**: Task history with multiple filter options
@ -124,20 +119,11 @@ Tape Backup
:align: right :align: right
:alt: Tape Backup: Tape changer overview :alt: Tape Backup: Tape changer overview
The `Tape Backup`_ section contains a top panel, with options for managing tape The `Tape Backup`_ section contains a top panel, managing tape media sets,
media sets, inventories, drives, changers, encryption keys, and the tape backup inventories, drives, changers and the tape backup jobs itself.
jobs itself. The tabs are as follows:
* **Content**: Information on the contents of the tape backup It also contains a subsection per standalone drive and per changer, with a
* **Inventory**: Manage the tapes attached to the system status and management view for those devices.
* **Changers**: Manage tape loading devices
* **Drives**: Manage drives used for reading and writing to tapes
* **Media Pools**: Manage logical pools of tapes
* **Encryption Keys**: Manage tape backup encryption keys
* **Backup Jobs**: Manage tape backup jobs
The section also contains a subsection per standalone drive and per changer,
with a status and management view for those devices.
Datastore Datastore
^^^^^^^^^ ^^^^^^^^^
@ -147,9 +133,9 @@ Datastore
:alt: Datastore Configuration :alt: Datastore Configuration
The Datastore section contains interfaces for creating and managing The Datastore section contains interfaces for creating and managing
datastores. It also contains a button for creating a new datastore on the datastores. It contains a button to create a new datastore on the server, as
server, as well as a subsection for each datastore on the system, in which you well as a subsection for each datastore on the system, in which you can use the
can use the top panel to view: top panel to view:
* **Summary**: Access a range of datastore usage statistics * **Summary**: Access a range of datastore usage statistics
* **Content**: Information on the datastore's backup groups and their respective * **Content**: Information on the datastore's backup groups and their respective
@ -158,7 +144,5 @@ can use the top panel to view:
collection <client_garbage-collection>` operations, and run garbage collection collection <client_garbage-collection>` operations, and run garbage collection
manually manually
* **Sync Jobs**: Create, manage and run :ref:`syncjobs` from remote servers * **Sync Jobs**: Create, manage and run :ref:`syncjobs` from remote servers
* **Verify Jobs**: Create, manage and run :ref:`maintenance_verification` jobs * **Verify Jobs**: Create, manage and run :ref:`maintenance_verification` jobs on the
on the datastore datastore
* **Options**: Configure notification and verification settings
* **Permissions**: Manage permissions on the datastore

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 149 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 438 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 197 KiB

After

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 104 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 367 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 83 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 131 KiB

After

Width:  |  Height:  |  Size: 130 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 139 KiB

After

Width:  |  Height:  |  Size: 79 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 174 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 33 KiB

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 107 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 132 KiB

After

Width:  |  Height:  |  Size: 62 KiB

View File

@ -50,7 +50,6 @@ in the section entitled "GNU Free Documentation License".
file-formats.rst file-formats.rst
backup-protocol.rst backup-protocol.rst
calendarevents.rst calendarevents.rst
markdown-primer.rst
glossary.rst glossary.rst
GFDL.rst GFDL.rst

View File

@ -19,24 +19,24 @@ for various management tasks such as disk management.
`Proxmox Backup`_ without the server part. `Proxmox Backup`_ without the server part.
The disk image (ISO file) provided by Proxmox includes a complete Debian system The disk image (ISO file) provided by Proxmox includes a complete Debian system
as well as all necessary packages for the `Proxmox Backup`_ Server. ("buster" for version 1.x) as well as all necessary packages for the `Proxmox Backup`_ server.
The installer will guide you through the setup process and allow The installer will guide you through the setup process and allow
you to partition the local disk(s), apply basic system configuration you to partition the local disk(s), apply basic system configurations
(for example timezone, language, network), and install all required packages. (e.g. timezone, language, network), and install all required packages.
The provided ISO will get you started in just a few minutes, and is the The provided ISO will get you started in just a few minutes, and is the
recommended method for new and existing users. recommended method for new and existing users.
Alternatively, `Proxmox Backup`_ Server can be installed on top of an Alternatively, `Proxmox Backup`_ server can be installed on top of an
existing Debian system. existing Debian system.
Install `Proxmox Backup`_ Server using the Installer Install `Proxmox Backup`_ with the Installer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Download the ISO from |DOWNLOADS|. Download the ISO from |DOWNLOADS|.
It includes the following: It includes the following:
* The `Proxmox Backup`_ Server installer, which partitions the local * The `Proxmox Backup`_ server installer, which partitions the local
disk(s) with ext4, xfs or ZFS, and installs the operating system disk(s) with ext4, xfs or ZFS, and installs the operating system
* Complete operating system (Debian Linux, 64-bit) * Complete operating system (Debian Linux, 64-bit)
@ -63,7 +63,7 @@ standard Debian installation. After configuring the
# apt-get update # apt-get update
# apt-get install proxmox-backup-server # apt-get install proxmox-backup-server
The above commands keep the current (Debian) kernel and install a minimal The commands above keep the current (Debian) kernel and install a minimal
set of required packages. set of required packages.
If you want to install the same set of packages as the installer If you want to install the same set of packages as the installer

View File

@ -4,16 +4,15 @@ Introduction
What is Proxmox Backup Server? What is Proxmox Backup Server?
------------------------------ ------------------------------
Proxmox Backup Server is an enterprise-class, client-server backup solution that Proxmox Backup Server is an enterprise-class, client-server backup software
is capable of backing up :term:`virtual machine<Virtual machine>`\ s, package that backs up :term:`virtual machine`\ s, :term:`container`\ s, and
:term:`container<Container>`\ s, and physical hosts. It is specially optimized physical hosts. It is specially optimized for the `Proxmox Virtual Environment`_
for the `Proxmox Virtual Environment`_ platform and allows you to back up your platform and allows you to back up your data securely, even between remote
data securely, even between remote sites, providing easy management through a sites, providing easy management with a web-based user interface.
web-based user interface.
It supports deduplication, compression, and authenticated It supports deduplication, compression, and authenticated
encryption (AE_). Using :term:`Rust` as the implementation language guarantees encryption (AE_). Using :term:`Rust` as the implementation language guarantees high
high performance, low resource usage, and a safe, high-quality codebase. performance, low resource usage, and a safe, high-quality codebase.
Proxmox Backup uses state of the art cryptography for both client-server Proxmox Backup uses state of the art cryptography for both client-server
communication and backup content :ref:`encryption <client_encryption>`. All communication and backup content :ref:`encryption <client_encryption>`. All
@ -29,24 +28,23 @@ Proxmox Backup Server uses a `client-server model`_. The server stores the
backup data and provides an API to create and manage datastores. With the backup data and provides an API to create and manage datastores. With the
API, it's also possible to manage disks and other server-side resources. API, it's also possible to manage disks and other server-side resources.
The backup client uses this API to access the backed up data. You can use the The backup client uses this API to access the backed up data. With the command
``proxmox-backup-client`` command line tool to create and restore file backups. line tool ``proxmox-backup-client`` you can create backups and restore data.
For QEMU_ and LXC_ within `Proxmox Virtual Environment`_, we deliver an For QEMU_ with `Proxmox Virtual Environment`_ we deliver an integrated client.
integrated client.
A single backup is allowed to contain several archives. For example, when you A single backup is allowed to contain several archives. For example, when you
backup a :term:`virtual machine<Virtual machine>`, each disk is stored as a backup a :term:`virtual machine`, each disk is stored as a separate archive
separate archive inside that backup. The VM configuration itself is stored as inside that backup. The VM configuration itself is stored as an extra file.
an extra file. This way, it's easy to access and restore only the important This way, it's easy to access and restore only important parts of the backup,
parts of the backup, without the need to scan the whole backup. without the need to scan the whole backup.
Main Features Main Features
------------- -------------
:Support for Proxmox VE: The `Proxmox Virtual Environment`_ is fully :Support for Proxmox VE: The `Proxmox Virtual Environment`_ is fully
supported, and you can easily backup :term:`virtual machine<Virtual machine>`\ s and supported and you can easily backup :term:`virtual machine`\ s and
:term:`container<Container>`\ s. :term:`container`\ s.
:Performance: The whole software stack is written in :term:`Rust`, :Performance: The whole software stack is written in :term:`Rust`,
in order to provide high speed and memory efficiency. in order to provide high speed and memory efficiency.
@ -72,10 +70,6 @@ Main Features
modern hardware. In addition to client-side encryption, all data is modern hardware. In addition to client-side encryption, all data is
transferred via a secure TLS connection. transferred via a secure TLS connection.
:Tape backup: For long-term archiving of data, Proxmox Backup Server also
provides extensive support for backing up to tape and managing tape
libraries.
:Web interface: Manage the Proxmox Backup Server with the integrated, web-based :Web interface: Manage the Proxmox Backup Server with the integrated, web-based
user interface. user interface.
@ -86,7 +80,7 @@ Main Features
backup-clients. backup-clients.
:Enterprise Support: Proxmox Server Solutions GmbH offers enterprise support in :Enterprise Support: Proxmox Server Solutions GmbH offers enterprise support in
the form of `Proxmox Backup Server Subscription Plans form of `Proxmox Backup Server Subscription Plans
<https://www.proxmox.com/en/proxmox-backup-server/pricing>`_. Users at every <https://www.proxmox.com/en/proxmox-backup-server/pricing>`_. Users at every
subscription level get access to the Proxmox Backup :ref:`Enterprise subscription level get access to the Proxmox Backup :ref:`Enterprise
Repository <sysadmin_package_repos_enterprise>`. In addition, with a Basic, Repository <sysadmin_package_repos_enterprise>`. In addition, with a Basic,
@ -179,7 +173,7 @@ Bug Tracker
~~~~~~~~~~~ ~~~~~~~~~~~
Proxmox runs a public bug tracker at `<https://bugzilla.proxmox.com>`_. If an Proxmox runs a public bug tracker at `<https://bugzilla.proxmox.com>`_. If an
issue appears, file your report there. An issue can be a bug, as well as a issue appears, file your report there. An issue can be a bug as well as a
request for a new feature or enhancement. The bug tracker helps to keep track request for a new feature or enhancement. The bug tracker helps to keep track
of the issue and will send a notification once it has been solved. of the issue and will send a notification once it has been solved.
@ -230,6 +224,5 @@ requirements.
In July 2020, we released the first beta version of Proxmox Backup In July 2020, we released the first beta version of Proxmox Backup
Server, followed by the first stable version in November 2020. With support for Server, followed by the first stable version in November 2020. With support for
encryption and incremental, fully deduplicated backups, Proxmox Backup offers a incremental, fully deduplicated backups, Proxmox Backup significantly reduces
secure environment, which significantly reduces network load and saves valuable network load and saves valuable storage space.
storage space.

View File

@ -4,17 +4,17 @@
ZFS on Linux ZFS on Linux
------------ ------------
ZFS is a combined file system and logical volume manager, designed by ZFS is a combined file system and logical volume manager designed by
Sun Microsystems. There is no need to manually compile ZFS modules - all Sun Microsystems. There is no need to manually compile ZFS modules - all
packages are included. packages are included.
By using ZFS, it's possible to achieve maximum enterprise features with By using ZFS, it's possible to achieve maximum enterprise features with
low budget hardware, and also high performance systems by leveraging low budget hardware, but also high performance systems by leveraging
SSD caching or even SSD only setups. ZFS can replace expensive SSD caching or even SSD only setups. ZFS can replace cost intense
hardware raid cards with moderate CPU and memory load, combined with easy hardware raid cards by moderate CPU and memory load combined with easy
management. management.
General advantages of ZFS: General ZFS advantages
* Easy configuration and management with GUI and CLI. * Easy configuration and management with GUI and CLI.
* Reliable * Reliable
@ -34,18 +34,18 @@ General advantages of ZFS:
Hardware Hardware
~~~~~~~~~ ~~~~~~~~~
ZFS depends heavily on memory, so it's recommended to have at least 8GB to ZFS depends heavily on memory, so you need at least 8GB to start. In
start. In practice, use as much you can get for your hardware/budget. To prevent practice, use as much you can get for your hardware/budget. To prevent
data corruption, we recommend the use of high quality ECC RAM. data corruption, we recommend the use of high quality ECC RAM.
If you use a dedicated cache and/or log disk, you should use an If you use a dedicated cache and/or log disk, you should use an
enterprise class SSD (for example, Intel SSD DC S3700 Series). This can enterprise class SSD (e.g. Intel SSD DC S3700 Series). This can
increase the overall performance significantly. increase the overall performance significantly.
IMPORTANT: Do not use ZFS on top of a hardware controller which has its IMPORTANT: Do not use ZFS on top of hardware controller which has its
own cache management. ZFS needs to directly communicate with disks. An own cache management. ZFS needs to directly communicate with disks. An
HBA adapter or something like an LSI controller flashed in ``IT`` mode is HBA adapter is the way to go, or something like LSI controller flashed
recommended. in ``IT`` mode.
ZFS Administration ZFS Administration
@ -53,7 +53,7 @@ ZFS Administration
This section gives you some usage examples for common tasks. ZFS This section gives you some usage examples for common tasks. ZFS
itself is really powerful and provides many options. The main commands itself is really powerful and provides many options. The main commands
to manage ZFS are `zfs` and `zpool`. Both commands come with extensive to manage ZFS are `zfs` and `zpool`. Both commands come with great
manual pages, which can be read with: manual pages, which can be read with:
.. code-block:: console .. code-block:: console
@ -123,7 +123,7 @@ Create a new pool with cache (L2ARC)
It is possible to use a dedicated cache drive partition to increase It is possible to use a dedicated cache drive partition to increase
the performance (use SSD). the performance (use SSD).
For `<device>`, you can use multiple devices, as is shown in As `<device>` it is possible to use more devices, like it's shown in
"Create a new pool with RAID*". "Create a new pool with RAID*".
.. code-block:: console .. code-block:: console
@ -136,7 +136,7 @@ Create a new pool with log (ZIL)
It is possible to use a dedicated cache drive partition to increase It is possible to use a dedicated cache drive partition to increase
the performance (SSD). the performance (SSD).
For `<device>`, you can use multiple devices, as is shown in As `<device>` it is possible to use more devices, like it's shown in
"Create a new pool with RAID*". "Create a new pool with RAID*".
.. code-block:: console .. code-block:: console
@ -146,9 +146,8 @@ For `<device>`, you can use multiple devices, as is shown in
Add cache and log to an existing pool Add cache and log to an existing pool
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You can add cache and log devices to a pool after its creation. In this example, If you have a pool without cache and log. First partition the SSD in
we will use a single drive for both cache and log. First, you need to create 2 partition with `parted` or `gdisk`
2 partitions on the SSD with `parted` or `gdisk`
.. important:: Always use GPT partition tables. .. important:: Always use GPT partition tables.
@ -172,12 +171,12 @@ Changing a failed device
Changing a failed bootable device Changing a failed bootable device
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Depending on how Proxmox Backup was installed, it is either using `grub` or Depending on how Proxmox Backup was installed it is either using `grub` or `systemd-boot`
`systemd-boot` as a bootloader. as bootloader.
In either case, the first steps of copying the partition table, reissuing GUIDs The first steps of copying the partition table, reissuing GUIDs and replacing
and replacing the ZFS partition are the same. To make the system bootable from the ZFS partition are the same. To make the system bootable from the new disk,
the new disk, different steps are needed which depend on the bootloader in use. different steps are needed which depend on the bootloader in use.
.. code-block:: console .. code-block:: console
@ -191,12 +190,12 @@ With `systemd-boot`:
.. code-block:: console .. code-block:: console
# proxmox-boot-tool format <new ESP> # pve-efiboot-tool format <new disk's ESP>
# proxmox-boot-tool init <new ESP> # pve-efiboot-tool init <new disk's ESP>
.. NOTE:: `ESP` stands for EFI System Partition, which is setup as partition #2 on .. NOTE:: `ESP` stands for EFI System Partition, which is setup as partition #2 on
bootable disks setup by the `Proxmox Backup`_ installer. For details, see bootable disks setup by the {pve} installer since version 5.4. For details, see
:ref:`Setting up a new partition for use as synced ESP <systembooting-proxmox-boot-setup>`. xref:sysboot_systemd_boot_setup[Setting up a new partition for use as synced ESP].
With `grub`: With `grub`:
@ -208,31 +207,36 @@ Usually `grub.cfg` is located in `/boot/grub/grub.cfg`
# grub-mkconfig -o /path/to/grub.cfg # grub-mkconfig -o /path/to/grub.cfg
Activate e-mail notification Activate E-Mail Notification
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ZFS comes with an event daemon ``ZED``, which monitors events generated by the ZFS comes with an event daemon, which monitors events generated by the
ZFS kernel module. The daemon can also send emails on ZFS events like pool ZFS kernel module. The daemon can also send emails on ZFS events like
errors. Newer ZFS packages ship the daemon in a separate package ``zfs-zed``, pool errors. Newer ZFS packages ship the daemon in a separate package,
which should already be installed by default in `Proxmox Backup`_. and you can install it using `apt-get`:
You can configure the daemon via the file ``/etc/zfs/zed.d/zed.rc`` with your .. code-block:: console
favorite editor. The required setting for email notification is
``ZED_EMAIL_ADDR``, which is set to ``root`` by default. # apt-get install zfs-zed
To activate the daemon it is necessary to edit `/etc/zfs/zed.d/zed.rc` with your
favorite editor, and uncomment the `ZED_EMAIL_ADDR` setting:
.. code-block:: console .. code-block:: console
ZED_EMAIL_ADDR="root" ZED_EMAIL_ADDR="root"
Please note that `Proxmox Backup`_ forwards mails to `root` to the email address Please note Proxmox Backup forwards mails to `root` to the email address
configured for the root user. configured for the root user.
IMPORTANT: The only setting that is required is `ZED_EMAIL_ADDR`. All
other settings are optional.
Limit ZFS memory usage Limit ZFS Memory Usage
^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^
It is good to use at most 50 percent (which is the default) of the It is good to use at most 50 percent (which is the default) of the
system memory for ZFS ARC, to prevent performance degradation of the system memory for ZFS ARC to prevent performance shortage of the
host. Use your preferred editor to change the configuration in host. Use your preferred editor to change the configuration in
`/etc/modprobe.d/zfs.conf` and insert: `/etc/modprobe.d/zfs.conf` and insert:
@ -240,40 +244,25 @@ host. Use your preferred editor to change the configuration in
options zfs zfs_arc_max=8589934592 options zfs zfs_arc_max=8589934592
The above example limits the usage to 8 GiB ('8 * 2^30^'). This example setting limits the usage to 8GB.
.. IMPORTANT:: In case your desired `zfs_arc_max` value is lower than or equal .. IMPORTANT:: If your root file system is ZFS you must update your initramfs every time this value changes:
to `zfs_arc_min` (which defaults to 1/32 of the system memory), `zfs_arc_max`
will be ignored. Thus, for it to work in this case, you must set
`zfs_arc_min` to at most `zfs_arc_max - 1`. This would require updating the
configuration in `/etc/modprobe.d/zfs.conf`, with:
.. code-block:: console
options zfs zfs_arc_min=8589934591
options zfs zfs_arc_max=8589934592
This example setting limits the usage to 8 GiB ('8 * 2^30^') on
systems with more than 256 GiB of total memory, where simply setting
`zfs_arc_max` alone would not work.
.. IMPORTANT:: If your root file system is ZFS, you must update your initramfs
every time this value changes.
.. code-block:: console .. code-block:: console
# update-initramfs -u # update-initramfs -u
Swap on ZFS SWAP on ZFS
^^^^^^^^^^^ ^^^^^^^^^^^
Swap-space created on a zvol may cause some issues, such as blocking the Swap-space created on a zvol may generate some troubles, like blocking the
server or generating a high IO load. server or generating a high IO load, often seen when starting a Backup
to an external Storage.
We strongly recommend using enough memory, so that you normally do not We strongly recommend to use enough memory, so that you normally do not
run into low memory situations. Should you need or want to add swap, it is run into low memory situations. Should you need or want to add swap, it is
preferred to create a partition on a physical disk and use it as a swap device. preferred to create a partition on a physical disk and use it as swap device.
You can leave some space free for this purpose in the advanced options of the You can leave some space free for this purpose in the advanced options of the
installer. Additionally, you can lower the `swappiness` value. installer. Additionally, you can lower the `swappiness` value.
A good value for servers is 10: A good value for servers is 10:
@ -302,24 +291,21 @@ an editor of your choice and add the following line:
vm.swappiness = 100 The kernel will swap aggressively. vm.swappiness = 100 The kernel will swap aggressively.
==================== =============================================================== ==================== ===============================================================
ZFS compression ZFS Compression
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
To activate compression: To activate compression:
.. code-block:: console .. code-block:: console
# zpool set compression=lz4 <pool> # zpool set compression=lz4 <pool>
We recommend using the `lz4` algorithm, since it adds very little CPU overhead. We recommend using the `lz4` algorithm, since it adds very little CPU overhead.
Other algorithms such as `lzjb`, `zstd` and `gzip-N` (where `N` is an integer from `1-9` Other algorithms such as `lzjb` and `gzip-N` (where `N` is an integer `1-9` representing
representing the compression ratio, where 1 is fastest and 9 is best the compression ratio, 1 is fastest and 9 is best compression) are also available.
compression) are also available. Depending on the algorithm and how Depending on the algorithm and how compressible the data is, having compression enabled can even increase
compressible the data is, having compression enabled can even increase I/O I/O performance.
performance.
You can disable compression at any time with: You can disable compression at any time with:
.. code-block:: console .. code-block:: console
# zfs set compression=off <dataset> # zfs set compression=off <dataset>
@ -328,26 +314,26 @@ Only new blocks will be affected by this change.
.. _local_zfs_special_device: .. _local_zfs_special_device:
ZFS special device ZFS Special Device
^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^
Since version 0.8.0, ZFS supports `special` devices. A `special` device in a Since version 0.8.0 ZFS supports `special` devices. A `special` device in a
pool is used to store metadata, deduplication tables, and optionally small pool is used to store metadata, deduplication tables, and optionally small
file blocks. file blocks.
A `special` device can improve the speed of a pool consisting of slow spinning A `special` device can improve the speed of a pool consisting of slow spinning
hard disks with a lot of metadata changes. For example, workloads that involve hard disks with a lot of metadata changes. For example workloads that involve
creating, updating or deleting a large number of files will benefit from the creating, updating or deleting a large number of files will benefit from the
presence of a `special` device. ZFS datasets can also be configured to store presence of a `special` device. ZFS datasets can also be configured to store
small files on the `special` device, which can further improve the whole small files on the `special` device which can further improve the
performance. Use fast SSDs for the `special` device. performance. Use fast SSDs for the `special` device.
.. IMPORTANT:: The redundancy of the `special` device should match the one of the .. IMPORTANT:: The redundancy of the `special` device should match the one of the
pool, since the `special` device is a point of failure for the entire pool. pool, since the `special` device is a point of failure for the whole pool.
.. WARNING:: Adding a `special` device to a pool cannot be undone! .. WARNING:: Adding a `special` device to a pool cannot be undone!
To create a pool with `special` device and RAID-1: Create a pool with `special` device and RAID-1:
.. code-block:: console .. code-block:: console
@ -360,8 +346,8 @@ Adding a `special` device to an existing pool with RAID-1:
# zpool add <pool> special mirror <device1> <device2> # zpool add <pool> special mirror <device1> <device2>
ZFS datasets expose the `special_small_blocks=<size>` property. `size` can be ZFS datasets expose the `special_small_blocks=<size>` property. `size` can be
`0` to disable storing small file blocks on the `special` device, or a power of `0` to disable storing small file blocks on the `special` device or a power of
two in the range between `512B` to `128K`. After setting this property, new file two in the range between `512B` to `128K`. After setting the property new file
blocks smaller than `size` will be allocated on the `special` device. blocks smaller than `size` will be allocated on the `special` device.
.. IMPORTANT:: If the value for `special_small_blocks` is greater than or equal to .. IMPORTANT:: If the value for `special_small_blocks` is greater than or equal to
@ -369,10 +355,10 @@ blocks smaller than `size` will be allocated on the `special` device.
the `special` device, so be careful! the `special` device, so be careful!
Setting the `special_small_blocks` property on a pool will change the default Setting the `special_small_blocks` property on a pool will change the default
value of that property for all child ZFS datasets (for example, all containers value of that property for all child ZFS datasets (for example all containers
in the pool will opt in for small file blocks). in the pool will opt in for small file blocks).
Opt in for all files smaller than 4K-blocks pool-wide: Opt in for all file smaller than 4K-blocks pool-wide:
.. code-block:: console .. code-block:: console
@ -393,15 +379,10 @@ Opt out from small file blocks for a single dataset:
Troubleshooting Troubleshooting
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
Corrupt cache file Corrupted cachefile
""""""""""""""""""
`zfs-import-cache.service` imports ZFS pools using the ZFS cache file. If this In case of a corrupted ZFS cachefile, some volumes may not be mounted during
file becomes corrupted, the service won't be able to import the pools that it's boot until mounted manually later.
unable to read from it.
As a result, in case of a corrupted ZFS cache file, some volumes may not be
mounted during boot and must be mounted manually later.
For each pool, run: For each pool, run:
@ -409,13 +390,16 @@ For each pool, run:
# zpool set cachefile=/etc/zfs/zpool.cache POOLNAME # zpool set cachefile=/etc/zfs/zpool.cache POOLNAME
then, update the `initramfs` by running: and afterwards update the `initramfs` by running:
.. code-block:: console .. code-block:: console
# update-initramfs -u -k all # update-initramfs -u -k all
and finally, reboot the node. and finally reboot your node.
Sometimes the ZFS cachefile can get corrupted, and `zfs-import-cache.service`
doesn't import the pools that aren't present in the cachefile.
Another workaround to this problem is enabling the `zfs-import-scan.service`, Another workaround to this problem is enabling the `zfs-import-scan.service`,
which searches and imports pools via device scanning (usually slower). which searches and imports pools via device scanning (usually slower).

View File

@ -34,7 +34,17 @@
</style> </style>
<link rel="stylesheet" type="text/css" href="font-awesome/css/font-awesome.css"/> <link rel="stylesheet" type="text/css" href="font-awesome/css/font-awesome.css"/>
<script type="text/javascript" src="extjs/ext-all.js"></script> <script type="text/javascript" src="extjs/ext-all.js"></script>
<script type="text/javascript" src="lto-barcode-generator.js"></script>
<script type="text/javascript" src="code39.js"></script>
<script type="text/javascript" src="prefix-field.js"></script>
<script type="text/javascript" src="label-style.js"></script>
<script type="text/javascript" src="tape-type.js"></script>
<script type="text/javascript" src="paper-size.js"></script>
<script type="text/javascript" src="page-layout.js"></script>
<script type="text/javascript" src="page-calibration.js"></script>
<script type="text/javascript" src="label-list.js"></script>
<script type="text/javascript" src="label-setup.js"></script>
<script type="text/javascript" src="lto-barcode.js"></script>
</head> </head>
<body> <body>
</body> </body>

View File

@ -1,5 +1,7 @@
// for toolkit.js // FIXME: HACK! Makes scrolling in number spinner work again. fixed in ExtJS >= 6.1
function gettext(val) { return val; }; if (Ext.isFirefox) {
Ext.$eventNameMap.DOMMouseScroll = 'DOMMouseScroll';
}
function draw_labels(target_id, label_list, page_layout, calibration) { function draw_labels(target_id, label_list, page_layout, calibration) {
let max_labels = compute_max_labels(page_layout); let max_labels = compute_max_labels(page_layout);

View File

@ -14,15 +14,15 @@ following retention options are available:
``keep-hourly <N>`` ``keep-hourly <N>``
Keep backups for the last ``<N>`` hours. If there is more than one Keep backups for the last ``<N>`` hours. If there is more than one
backup for a single hour, only the latest is retained. backup for a single hour, only the latest is kept.
``keep-daily <N>`` ``keep-daily <N>``
Keep backups for the last ``<N>`` days. If there is more than one Keep backups for the last ``<N>`` days. If there is more than one
backup for a single day, only the latest is retained. backup for a single day, only the latest is kept.
``keep-weekly <N>`` ``keep-weekly <N>``
Keep backups for the last ``<N>`` weeks. If there is more than one Keep backups for the last ``<N>`` weeks. If there is more than one
backup for a single week, only the latest is retained. backup for a single week, only the latest is kept.
.. note:: Weeks start on Monday and end on Sunday. The software .. note:: Weeks start on Monday and end on Sunday. The software
uses the `ISO week date`_ system and handles weeks at uses the `ISO week date`_ system and handles weeks at
@ -30,17 +30,17 @@ following retention options are available:
``keep-monthly <N>`` ``keep-monthly <N>``
Keep backups for the last ``<N>`` months. If there is more than one Keep backups for the last ``<N>`` months. If there is more than one
backup for a single month, only the latest is retained. backup for a single month, only the latest is kept.
``keep-yearly <N>`` ``keep-yearly <N>``
Keep backups for the last ``<N>`` years. If there is more than one Keep backups for the last ``<N>`` years. If there is more than one
backup for a single year, only the latest is retained. backup for a single year, only the latest is kept.
The retention options are processed in the order given above. Each option The retention options are processed in the order given above. Each option
only covers backups within its time period. The next option does not take care only covers backups within its time period. The next option does not take care
of already covered backups. It will only consider older backups. of already covered backups. It will only consider older backups.
Unfinished and incomplete backups will be removed by the prune command, unless Unfinished and incomplete backups will be removed by the prune command unless
they are newer than the last successful backup. In this case, the last failed they are newer than the last successful backup. In this case, the last failed
backup is retained. backup is retained.
@ -48,7 +48,7 @@ Prune Simulator
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
You can use the built-in `prune simulator <prune-simulator/index.html>`_ You can use the built-in `prune simulator <prune-simulator/index.html>`_
to explore the effect of different retention options with various backup to explore the effect of different retetion options with various backup
schedules. schedules.
Manual Pruning Manual Pruning
@ -59,10 +59,10 @@ Manual Pruning
:align: right :align: right
:alt: Prune and garbage collection options :alt: Prune and garbage collection options
To manually prune a specific backup group, you can use To access pruning functionality for a specific backup group, you can use the
``proxmox-backup-client``'s ``prune`` subcommand, discussed in prune command line option discussed in :ref:`backup-pruning`, or navigate to
:ref:`backup-pruning`, or navigate to the **Content** tab of the datastore and the **Content** tab of the datastore and click the scissors icon in the
click the scissors icon in the **Actions** column of the relevant backup group. **Actions** column of the relevant backup group.
Prune Schedules Prune Schedules
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
@ -81,7 +81,7 @@ Retention Settings Example
^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
The backup frequency and retention of old backups may depend on how often data The backup frequency and retention of old backups may depend on how often data
changes and how important an older state may be in a specific workload. changes, and how important an older state may be, in a specific work load.
When backups act as a company's document archive, there may also be legal When backups act as a company's document archive, there may also be legal
requirements for how long backup snapshots must be kept. requirements for how long backup snapshots must be kept.
@ -125,8 +125,8 @@ start garbage collection on an entire datastore and the ``status`` subcommand to
see attributes relating to the :ref:`garbage collection <client_garbage-collection>`. see attributes relating to the :ref:`garbage collection <client_garbage-collection>`.
This functionality can also be accessed in the GUI, by navigating to **Prune & This functionality can also be accessed in the GUI, by navigating to **Prune &
GC** from the top panel of a datastore. From here, you can edit the schedule at GC** from the top panel. From here, you can edit the schedule at which garbage
which garbage collection runs and manually start the operation. collection runs and manually start the operation.
.. _maintenance_verification: .. _maintenance_verification:
@ -139,13 +139,13 @@ Verification
:align: right :align: right
:alt: Adding a verify job :alt: Adding a verify job
Proxmox Backup Server offers various verification options to ensure that backup Proxmox Backup offers various verification options to ensure that backup data is
data is intact. Verification is generally carried out through the creation of intact. Verification is generally carried out through the creation of verify
verify jobs. These are scheduled tasks that run verification at a given interval jobs. These are scheduled tasks that run verification at a given interval (see
(see :ref:`calendar-event-scheduling`). With these, you can also set whether :ref:`calendar-event-scheduling`). With these, you can set whether already verified
already verified snapshots are ignored, as well as set a time period, after snapshots are ignored, as well as set a time period, after which verified jobs
which snapshots are checked again. The interface for creating verify jobs can be are checked again. The interface for creating verify jobs can be found under the
found under the **Verify Jobs** tab of the datastore. **Verify Jobs** tab of the datastore.
.. Note:: It is recommended that you reverify all backups at least monthly, even .. Note:: It is recommended that you reverify all backups at least monthly, even
if a previous verification was successful. This is because physical drives if a previous verification was successful. This is because physical drives
@ -158,9 +158,9 @@ found under the **Verify Jobs** tab of the datastore.
data. data.
Aside from using verify jobs, you can also run verification manually on entire Aside from using verify jobs, you can also run verification manually on entire
datastores, backup groups or snapshots. To do this, navigate to the **Content** datastores, backup groups, or snapshots. To do this, navigate to the **Content**
tab of the datastore and either click *Verify All* or select the *V.* icon from tab of the datastore and either click *Verify All*, or select the *V.* icon from
the **Actions** column in the table. the *Actions* column in the table.
.. _maintenance_notification: .. _maintenance_notification:
@ -170,12 +170,8 @@ Notifications
Proxmox Backup Server can send you notification emails about automatically Proxmox Backup Server can send you notification emails about automatically
scheduled verification, garbage-collection and synchronization tasks results. scheduled verification, garbage-collection and synchronization tasks results.
By default, notifications are sent to the email address configured for the By default, notifications are send to the email address configured for the
`root@pam` user. You can instead set this user for each datastore. `root@pam` user. You can set that user for each datastore.
.. image:: images/screenshots/pbs-gui-datastore-options.png
:align: right
:alt: Datastore Options
You can also change the level of notification received per task type, the You can also change the level of notification received per task type, the
following options are available: following options are available:
@ -183,23 +179,6 @@ following options are available:
* Always: send a notification for any scheduled task, independent of the * Always: send a notification for any scheduled task, independent of the
outcome outcome
* Errors: send a notification for any scheduled task that results in an error * Errors: send a notification for any scheduled task resulting in an error
* Never: do not send any notification at all * Never: do not send any notification at all
.. _maintenance_mode:
Maintenance Mode
----------------
Proxmox Backup Server implements setting the `read-only` and `offline`
maintenance modes for a datastore.
Once enabled, depending on the mode, new reads and/or writes to the datastore
are blocked, allowing an administrator to safely execute maintenance tasks, for
example, on the underlying storage.
Internally Proxmox Backup Server tracks whether each datastore access is a
write or read operation, so that it can gracefully enter the respective mode,
by allowing conflicting operations that started before enabling the maintenance
mode to finish.

View File

@ -1,5 +1,5 @@
Managing Remotes & Sync Managing Remotes
======================= ================
.. _backup_remote: .. _backup_remote:
@ -17,8 +17,8 @@ configuration information for remotes is stored in the file
:align: right :align: right
:alt: Add a remote :alt: Add a remote
To add a remote, you need its hostname or IP address, a userid and password on To add a remote, you need its hostname or IP, a userid and password on the
the remote, and its certificate fingerprint. To get the fingerprint, use the remote, and its certificate fingerprint. To get the fingerprint, use the
``proxmox-backup-manager cert info`` command on the remote, or navigate to ``proxmox-backup-manager cert info`` command on the remote, or navigate to
**Dashboard** in the remote's web interface and select **Show Fingerprint**. **Dashboard** in the remote's web interface and select **Show Fingerprint**.
@ -60,13 +60,12 @@ Sync Jobs
Sync jobs are configured to pull the contents of a datastore on a **Remote** to Sync jobs are configured to pull the contents of a datastore on a **Remote** to
a local datastore. You can manage sync jobs in the web interface, from the a local datastore. You can manage sync jobs in the web interface, from the
**Sync Jobs** tab of the **Datastore** panel or from that of the Datastore **Sync Jobs** tab of the datastore which you'd like to set one up for, or using
itself. Alternatively, you can manage them with the ``proxmox-backup-manager the ``proxmox-backup-manager sync-job`` command. The configuration information
sync-job`` command. The configuration information for sync jobs is stored at for sync jobs is stored at ``/etc/proxmox-backup/sync.cfg``. To create a new
``/etc/proxmox-backup/sync.cfg``. To create a new sync job, click the add button sync job, click the add button in the GUI, or use the ``create`` subcommand.
in the GUI, or use the ``create`` subcommand. After creating a sync job, you can After creating a sync job, you can either start it manually from the GUI or
either start it manually from the GUI or provide it with a schedule (see provide it with a schedule (see :ref:`calendar-event-scheduling`) to run regularly.
:ref:`calendar-event-scheduling`) to run regularly.
.. code-block:: console .. code-block:: console
@ -80,130 +79,17 @@ either start it manually from the GUI or provide it with a schedule (see
└────────────┴───────┴────────┴──────────────┴───────────┴─────────┘ └────────────┴───────┴────────┴──────────────┴───────────┴─────────┘
# proxmox-backup-manager sync-job remove pbs2-local # proxmox-backup-manager sync-job remove pbs2-local
To set up sync jobs, the configuring user needs the following permissions: For setting up sync jobs, the configuring user needs the following permissions:
#. ``Remote.Read`` on the ``/remote/{remote}/{remote-store}`` path #. ``Remote.Read`` on the ``/remote/{remote}/{remote-store}`` path
#. At least ``Datastore.Backup`` on the local target datastore (``/datastore/{store}``) #. at least ``Datastore.Backup`` on the local target datastore (``/datastore/{store}``)
If the ``remove-vanished`` option is set, ``Datastore.Prune`` is required on
the local datastore as well. If the ``owner`` option is not set (defaulting to
``root@pam``) or set to something other than the configuring user,
``Datastore.Modify`` is required as well.
.. note:: A sync job can only sync backup groups that the configured remote's .. note:: A sync job can only sync backup groups that the configured remote's
user/API token can read. If a remote is configured with a user/API token that user/API token can read. If a remote is configured with a user/API token that
only has ``Datastore.Backup`` privileges, only the limited set of accessible only has ``Datastore.Backup`` privileges, only the limited set of accessible
snapshots owned by that user/API token can be synced. snapshots owned by that user/API token can be synced.
If the ``remove-vanished`` option is set, ``Datastore.Prune`` is required on
the local datastore as well. If the ``owner`` option is not set (defaulting to
``root@pam``) or is set to something other than the configuring user,
``Datastore.Modify`` is required as well.
If the ``group-filter`` option is set, only backup groups matching at least one
of the specified criteria are synced. The available criteria are:
* backup type, for example to only sync groups of the `ct` (Container) type:
.. code-block:: console
# proxmox-backup-manager sync-job update ID --group-filter type:ct
* full group identifier
.. code-block:: console
# proxmox-backup-manager sync-job update ID --group-filter group:vm/100
* regular expression matched against the full group identifier
.. todo:: add example for regex
The same filter is applied to local groups for handling of the
``remove-vanished`` option.
.. note:: The ``protected`` flag of remote backup snapshots will not be synced.
Namespace Support
^^^^^^^^^^^^^^^^^
Sync jobs can be configured to not only sync datastores, but also sub-sets of
datastores in the form of namespaces or namespace sub-trees. The following
parameters influence how namespaces are treated as part of a sync job
execution:
- ``remote-ns``: the remote namespace anchor (default: the root namespace)
- ``ns``: the local namespace anchor (default: the root namespace)
- ``max-depth``: whether to recursively iterate over sub-namespaces of the remote
namespace anchor (default: `None`)
If ``max-depth`` is set to `0`, groups are synced from ``remote-ns`` into
``ns``, without any recursion. If it is set to `None` (left empty), recursion
depth will depend on the value of ``remote-ns`` and the remote side's
availability of namespace support:
- ``remote-ns`` set to something other than the root namespace: remote *must*
support namespaces, full recursion starting at ``remote-ns``.
- ``remote-ns`` set to root namespace and remote *supports* namespaces: full
recursion starting at root namespace.
- ``remote-ns`` set to root namespace and remote *does not support* namespaces:
backwards-compat mode, only root namespace will be synced into ``ns``, no
recursion.
Any other value of ``max-depth`` will limit recursion to at most ``max-depth``
levels, for example: ``remote-ns`` set to `location_a/department_b` and
``max-depth`` set to `1` will result in `location_a/department_b` and at most
one more level of sub-namespaces being synced.
The namespace tree starting at ``remote-ns`` will be mapped into ``ns`` up to a
depth of ``max-depth``.
For example, with the following namespaces at the remote side:
- `location_a`
- `location_a/department_x`
- `location_a/department_x/team_one`
- `location_a/department_x/team_two`
- `location_a/department_y`
- `location_a/department_y/team_one`
- `location_a/department_y/team_two`
- `location_b`
and ``remote-ns`` being set to `location_a/department_x` and ``ns`` set to
`location_a_dep_x` resulting in the following namespace tree on the sync
target:
- `location_a_dep_x` (containing the remote's `location_a/department_x`)
- `location_a_dep_x/team_one` (containing the remote's `location_a/department_x/team_one`)
- `location_a_dep_x/team_two` (containing the remote's `location_a/department_x/team_two`)
with the rest of the remote namespaces and groups not being synced (by this
sync job).
If a remote namespace is included in the sync job scope, but does not exist
locally, it will be created (provided the sync job owner has sufficient
privileges).
If the ``remove-vanished`` option is set, namespaces that are included in the
sync job scope but only exist locally are treated as vanished and removed
(provided the sync job owner has sufficient privileges).
.. note:: All other limitations on sync scope (such as remote user/API token
privileges, group filters) also apply for sync jobs involving one or
multiple namespaces.
Bandwidth Limit
^^^^^^^^^^^^^^^
Syncing a datastore to an archive can produce lots of traffic and impact other
users of the network. So, to avoid network or storage congestion you can limit
the bandwidth of the sync job by setting the ``rate-in`` option either in the
web interface or using the ``proxmox-backup-manager`` command-line tool:
.. code-block:: console
# proxmox-backup-manager sync-job update ID --rate-in 20MiB

View File

@ -1,178 +0,0 @@
.. _markdown-primer:
Markdown Primer
===============
"Markdown is a text-to-HTML conversion tool for web writers. Markdown allows
you to write using an easy-to-read, easy-to-write plain text format, then
convertit to structurally valid XHTML (or HTML)."
-- John Gruber, https://daringfireball.net/projects/markdown/
The Proxmox Backup Server (PBS) web-interface has support for using Markdown to
rendering rich text formatting in node and virtual guest notes.
PBS supports CommonMark with most extensions of GFM (GitHub Flavoured Markdown),
like tables or task-lists.
.. _markdown_basics:
Markdown Basics
---------------
Note that we only describe the basics here, please search the web for more
extensive resources, for example on https://www.markdownguide.org/
Headings
~~~~~~~~
.. code-block:: md
# This is a Heading h1
## This is a Heading h2
##### This is a Heading h5
Emphasis
~~~~~~~~
Use ``*text*`` or ``_text_`` for emphasis.
Use ``**text**`` or ``__text__`` for bold, heavy-weight text.
Combinations are also possible, for example:
.. code-block:: md
_You **can** combine them_
Links
~~~~~
You can use automatic detection of links, for example,
``https://forum.proxmox.com/`` would transform it into a clickable link.
You can also control the link text, for example:
.. code-block:: md
Now, [the part in brackets will be the link text](https://forum.proxmox.com/).
Lists
~~~~~
Unordered Lists
^^^^^^^^^^^^^^^
Use ``*`` or ``-`` for unordered lists, for example:
.. code-block:: md
* Item 1
* Item 2
* Item 2a
* Item 2b
Adding an indentation can be used to created nested lists.
Ordered Lists
^^^^^^^^^^^^^
.. code-block:: md
1. Item 1
1. Item 2
1. Item 3
1. Item 3a
1. Item 3b
NOTE: The integer of ordered lists does not need to be correct, they will be numbered automatically.
Task Lists
^^^^^^^^^^
Task list use a empty box ``[ ]`` for unfinished tasks and a box with an `X` for finished tasks.
For example:
.. code-block:: md
- [X] First task already done!
- [X] Second one too
- [ ] This one is still to-do
- [ ] So is this one
Tables
~~~~~~
Tables use the pipe symbol ``|`` to separate columns, and ``-`` to separate the
table header from the table body, in that separation one can also set the text
alignment, making one column left-, center-, or right-aligned.
.. code-block:: md
| Left columns | Right columns | Some | More | Cols.| Centering Works Too
| ------------- |--------------:|--------|------|------|:------------------:|
| left foo | right foo | First | Row | Here | >center< |
| left bar | right bar | Second | Row | Here | 12345 |
| left baz | right baz | Third | Row | Here | Test |
| left zab | right zab | Fourth | Row | Here | ☁️☁️☁️ |
| left rab | right rab | And | Last | Here | The End |
Note that you do not need to align the columns nicely with white space, but that makes
editing tables easier.
Block Quotes
~~~~~~~~~~~~
You can enter block quotes by prefixing a line with ``>``, similar as in plain-text emails.
.. code-block:: md
> Markdown is a lightweight markup language with plain-text-formatting syntax,
> created in 2004 by John Gruber with Aaron Swartz.
>
>> Markdown is often used to format readme files, for writing messages in online discussion forums,
>> and to create rich text using a plain text editor.
Code and Snippets
~~~~~~~~~~~~~~~~~
You can use backticks to avoid processing for a few word or paragraphs. That is useful for
avoiding that a code or configuration hunk gets mistakenly interpreted as markdown.
Inline code
^^^^^^^^^^^
Surrounding part of a line with single backticks allows to write code inline,
for examples:
.. code-block:: md
This hosts IP address is `10.0.0.1`.
Whole blocks of code
^^^^^^^^^^^^^^^^^^^^
For code blocks spanning several lines you can use triple-backticks to start
and end such a block, for example:
.. code-block:: md
```
# This is the network config I want to remember here
auto vmbr2
iface vmbr2 inet static
address 10.0.0.1/24
bridge-ports ens20
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
```

View File

@ -3,10 +3,6 @@
Network Management Network Management
================== ==================
.. image:: images/screenshots/pbs-gui-system-config.png
:align: right
:alt: System and Network Configuration Overview
Proxmox Backup Server provides both a web interface and a command line tool for Proxmox Backup Server provides both a web interface and a command line tool for
network configuration. You can find the configuration options in the web network configuration. You can find the configuration options in the web
interface under the **Network Interfaces** section of the **Configuration** menu interface under the **Network Interfaces** section of the **Configuration** menu
@ -35,6 +31,10 @@ To get a list of available interfaces, use the following command:
│ ens19 │ eth │ 1 │ manual │ │ │ │ │ ens19 │ eth │ 1 │ manual │ │ │ │
└───────┴────────┴───────────┴────────┴─────────────┴──────────────┴──────────────┘ └───────┴────────┴───────────┴────────┴─────────────┴──────────────┴──────────────┘
.. image:: images/screenshots/pbs-gui-network-create-bond.png
:align: right
:alt: Add a network interface
To add a new network interface, use the ``create`` subcommand with the relevant To add a new network interface, use the ``create`` subcommand with the relevant
parameters. For example, you may want to set up a bond, for the purpose of parameters. For example, you may want to set up a bond, for the purpose of
network redundancy. The following command shows a template for creating the bond shown network redundancy. The following command shows a template for creating the bond shown
@ -44,10 +44,6 @@ in the list above:
# proxmox-backup-manager network create bond0 --type bond --bond_mode active-backup --slaves ens18,ens19 --autostart true --cidr x.x.x.x/x --gateway x.x.x.x # proxmox-backup-manager network create bond0 --type bond --bond_mode active-backup --slaves ens18,ens19 --autostart true --cidr x.x.x.x/x --gateway x.x.x.x
.. image:: images/screenshots/pbs-gui-network-create-bond.png
:align: right
:alt: Add a network interface
You can make changes to the configuration of a network interface with the You can make changes to the configuration of a network interface with the
``update`` subcommand: ``update`` subcommand:
@ -86,12 +82,9 @@ is:
.. note:: This command and corresponding GUI button rely on the ``ifreload`` .. note:: This command and corresponding GUI button rely on the ``ifreload``
command, from the package ``ifupdown2``. This package is included within the command, from the package ``ifupdown2``. This package is included within the
Proxmox Backup Server installation, however, you may have to install it yourself, Proxmox Backup Server installation, however, you may have to install it yourself,
if you have installed Proxmox Backup Server on top of Debian or a Proxmox VE if you have installed Proxmox Backup Server on top of Debian or Proxmox VE.
version prior to version 7.
You can also configure DNS settings, from the **DNS** section You can also configure DNS settings, from the **DNS** section
of **Configuration** or by using the ``dns`` subcommand of of **Configuration** or by using the ``dns`` subcommand of
``proxmox-backup-manager``. ``proxmox-backup-manager``.
.. include:: traffic-control.rst

View File

@ -1,5 +1,5 @@
Most commands that produce output support the ``--output-format`` Most commands producing output supports the ``--output-format``
parameter. This accepts the following values: parameter. It accepts the following values:
:``text``: Text format (default). Structured data is rendered as a table. :``text``: Text format (default). Structured data is rendered as a table.

View File

@ -17,20 +17,18 @@ update``.
.. code-block:: sources.list .. code-block:: sources.list
:caption: File: ``/etc/apt/sources.list`` :caption: File: ``/etc/apt/sources.list``
deb http://ftp.debian.org/debian bullseye main contrib deb http://ftp.debian.org/debian buster main contrib
deb http://ftp.debian.org/debian bullseye-updates main contrib deb http://ftp.debian.org/debian buster-updates main contrib
# security updates # security updates
deb http://security.debian.org/debian-security bullseye-security main contrib deb http://security.debian.org/debian-security buster/updates main contrib
.. FIXME for 7.0: change security update suite to bullseye-security
In addition, you need a package repository from Proxmox to get Proxmox Backup In addition, you need a package repository from Proxmox to get Proxmox Backup
updates. updates.
.. image:: images/screenshots/pbs-gui-administration-apt-repos.png
:align: right
:alt: APT Repository Management in the Web Interface
.. _package_repos_secure_apt: .. _package_repos_secure_apt:
SecureApt SecureApt
@ -47,21 +45,31 @@ key with the following commands:
.. code-block:: console .. code-block:: console
# wget https://enterprise.proxmox.com/debian/proxmox-release-bullseye.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg # wget http://download.proxmox.com/debian/proxmox-ve-release-6.x.gpg -O /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg
Verify the SHA512 checksum afterwards with the expected output below: Verify the SHA512 checksum afterwards with:
.. code-block:: console .. code-block:: console
# sha512sum /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg # sha512sum /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg
7fb03ec8a1675723d2853b84aa4fdb49a46a3bb72b9951361488bfd19b29aab0a789a4f8c7406e71a69aabbc727c936d3549731c4659ffa1a08f44db8fdcebfa /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg
and the md5sum, with the expected output below: The output should be:
.. code-block:: console .. code-block:: console
# md5sum /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg acca6f416917e8e11490a08a1e2842d500b3a5d9f322c6319db0927b2901c3eae23cfb5cd5df6facf2b57399d3cfa52ad7769ebdd75d9b204549ca147da52626 /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg
bcc35c7173e0845c0d6ad6470b70f50e /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg
and the md5sum:
.. code-block:: console
# md5sum /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg
Here, the output should be:
.. code-block:: console
f3f6c5a3a67baf38ad178e5ff1ee270c /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg
.. _sysadmin_package_repos_enterprise: .. _sysadmin_package_repos_enterprise:
@ -76,7 +84,7 @@ enabled by default:
.. code-block:: sources.list .. code-block:: sources.list
:caption: File: ``/etc/apt/sources.list.d/pbs-enterprise.list`` :caption: File: ``/etc/apt/sources.list.d/pbs-enterprise.list``
deb https://enterprise.proxmox.com/debian/pbs bullseye pbs-enterprise deb https://enterprise.proxmox.com/debian/pbs buster pbs-enterprise
To never miss important security fixes, the superuser (``root@pam`` user) is To never miss important security fixes, the superuser (``root@pam`` user) is
@ -106,15 +114,15 @@ We recommend to configure this repository in ``/etc/apt/sources.list``.
.. code-block:: sources.list .. code-block:: sources.list
:caption: File: ``/etc/apt/sources.list`` :caption: File: ``/etc/apt/sources.list``
deb http://ftp.debian.org/debian bullseye main contrib deb http://ftp.debian.org/debian buster main contrib
deb http://ftp.debian.org/debian bullseye-updates main contrib deb http://ftp.debian.org/debian buster-updates main contrib
# PBS pbs-no-subscription repository provided by proxmox.com, # PBS pbs-no-subscription repository provided by proxmox.com,
# NOT recommended for production use # NOT recommended for production use
deb http://download.proxmox.com/debian/pbs bullseye pbs-no-subscription deb http://download.proxmox.com/debian/pbs buster pbs-no-subscription
# security updates # security updates
deb http://security.debian.org/debian-security bullseye-security main contrib deb http://security.debian.org/debian-security buster/updates main contrib
`Proxmox Backup`_ Test Repository `Proxmox Backup`_ Test Repository
@ -132,7 +140,7 @@ You can access this repository by adding the following line to
.. code-block:: sources.list .. code-block:: sources.list
:caption: sources.list entry for ``pbstest`` :caption: sources.list entry for ``pbstest``
deb http://download.proxmox.com/debian/pbs bullseye pbstest deb http://download.proxmox.com/debian/pbs buster pbstest
.. _package_repositories_client_only: .. _package_repositories_client_only:
@ -153,26 +161,6 @@ APT-based Proxmox Backup Client Repository
For modern Linux distributions using `apt` as package manager, like all Debian For modern Linux distributions using `apt` as package manager, like all Debian
and Ubuntu Derivative do, you may be able to use the APT-based repository. and Ubuntu Derivative do, you may be able to use the APT-based repository.
In order to configure this repository you need to first :ref:`setup the Proxmox
release key <package_repos_secure_apt>`. After that, add the repository URL to
the APT sources lists.
**Repositories for Debian 11 (Bullseye) based releases**
This repository is tested with:
- Debian Bullseye
Edit the file ``/etc/apt/sources.list.d/pbs-client.list`` and add the following
snipped
.. code-block:: sources.list
:caption: File: ``/etc/apt/sources.list``
deb http://download.proxmox.com/debian/pbs-client bullseye main
**Repositories for Debian 10 (Buster) based releases**
This repository is tested with: This repository is tested with:
- Debian Buster - Debian Buster
@ -180,6 +168,9 @@ This repository is tested with:
It may work with older, and should work with more recent released versions. It may work with older, and should work with more recent released versions.
In order to configure this repository you need to first :ref:`setup the Proxmox
release key <package_repos_secure_apt>`. After that, add the repository URL to
the APT sources lists.
Edit the file ``/etc/apt/sources.list.d/pbs-client.list`` and add the following Edit the file ``/etc/apt/sources.list.d/pbs-client.list`` and add the following
snipped snipped

View File

@ -51,7 +51,7 @@ ENVIRONMENT
:CHANGER: If set, replaces the `--device` option :CHANGER: If set, replaces the `--device` option
:PROXMOX_TAPE_DRIVE: If set, use the Proxmox Backup Server :PROXMOX_TAPE_DRIVE: If set, use the Proxmox Backup Server
configuration to find the associated changer device. configuration to find the associcated changer device.
.. include:: ../pbs-copyright.rst .. include:: ../pbs-copyright.rst

View File

@ -1,14 +0,0 @@
Implements debugging functionality to inspect Proxmox Backup datastore
files, verify the integrity of chunks.
Also contains an 'api' subcommand where arbitrary api paths can be called
(get/create/set/delete) as well as display their parameters (usage) and
their child-links (ls).
By default, it connects to the proxmox-backup-proxy on localhost via https,
but by setting the environment variable `PROXMOX_DEBUG_API_CODE` to `1` the
tool directly calls the corresponding code.
.. WARNING:: Using `PROXMOX_DEBUG_API_CODE` can be dangerous and is only intended
for debugging purposes. It is not intended for use on a production system.

View File

@ -1,33 +0,0 @@
==========================
proxmox-backup-debug
==========================
.. include:: ../epilog.rst
-------------------------------------------------------------
Debugging command line tool for Backup and Restore
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 1
Synopsis
==========
.. include:: synopsis.rst
Common Options
==============
.. include:: ../output-format.rst
Description
============
.. include:: description.rst
.. include:: ../pbs-copyright.rst

View File

@ -1,5 +1,5 @@
This daemon exposes the whole Proxmox Backup Server API on TCP port This daemon exposes the whole Proxmox Backup Server API on TCP port
8007 using HTTPS. It runs as user ``backup`` and has very limited 8007 using HTTPS. It runs as user ``backup`` and has very limited
permissions. Operations requiring more permissions are forwarded to permissions. Operation requiring more permissions are forwarded to
the local ``proxmox-backup`` service. the local ``proxmox-backup`` service.

View File

@ -1,5 +1,7 @@
// for Toolkit.js // FIXME: HACK! Makes scrolling in number spinner work again. fixed in ExtJS >= 6.1
function gettext(val) { return val; }; if (Ext.isFirefox) {
Ext.$eventNameMap.DOMMouseScroll = 'DOMMouseScroll';
}
Ext.onReady(function() { Ext.onReady(function() {
const NOW = new Date(); const NOW = new Date();
@ -35,6 +37,7 @@ Ext.onReady(function() {
editable: true, editable: true,
displayField: 'text',
valueField: 'value', valueField: 'value',
queryMode: 'local', queryMode: 'local',

View File

@ -3,8 +3,8 @@
`Proxmox VE`_ Integration `Proxmox VE`_ Integration
------------------------- -------------------------
Proxmox Backup Server can be integrated into a Proxmox VE standalone or cluster A Proxmox Backup Server can be integrated into a Proxmox VE setup by adding the
setup, by adding it as a storage in Proxmox VE. former as a storage in a Proxmox VE standalone or cluster setup.
See also the `Proxmox VE Storage - Proxmox Backup Server See also the `Proxmox VE Storage - Proxmox Backup Server
<https://pve.proxmox.com/pve-docs/pve-admin-guide.html#storage_pbs>`_ section <https://pve.proxmox.com/pve-docs/pve-admin-guide.html#storage_pbs>`_ section
@ -14,8 +14,8 @@ of the Proxmox VE Administration Guide for Proxmox VE specific documentation.
Using the Proxmox VE Web-Interface Using the Proxmox VE Web-Interface
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Proxmox VE has native API and web interface integration of Proxmox Backup Proxmox VE has native API and web-interface integration of Proxmox Backup
Server as of `Proxmox VE 6.3 Server since the `Proxmox VE 6.3 release
<https://pve.proxmox.com/wiki/Roadmap#Proxmox_VE_6.3>`_. <https://pve.proxmox.com/wiki/Roadmap#Proxmox_VE_6.3>`_.
A Proxmox Backup Server can be added under ``Datacenter -> Storage``. A Proxmox Backup Server can be added under ``Datacenter -> Storage``.
@ -24,8 +24,8 @@ Using the Proxmox VE Command-Line
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You need to define a new storage with type 'pbs' on your `Proxmox VE`_ You need to define a new storage with type 'pbs' on your `Proxmox VE`_
node. The following example uses ``store2`` as the storage's name, and node. The following example uses ``store2`` as storage name, and
assumes the server address is ``localhost`` and you want to connect assumes the server address is ``localhost``, and you want to connect
as ``user1@pbs``. as ``user1@pbs``.
.. code-block:: console .. code-block:: console
@ -33,7 +33,7 @@ as ``user1@pbs``.
# pvesm add pbs store2 --server localhost --datastore store2 # pvesm add pbs store2 --server localhost --datastore store2
# pvesm set store2 --username user1@pbs --password <secret> # pvesm set store2 --username user1@pbs --password <secret>
.. note:: If you would rather not enter your password as plain text, you can pass .. note:: If you would rather not pass your password as plain text, you can pass
the ``--password`` parameter, without any arguments. This will cause the the ``--password`` parameter, without any arguments. This will cause the
program to prompt you for a password upon entering the command. program to prompt you for a password upon entering the command.
@ -53,7 +53,7 @@ relationship:
# pvesm set store2 --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe # pvesm set store2 --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
After that, you should be able to view storage status with: After that you should be able to see storage status with:
.. code-block:: console .. code-block:: console

View File

@ -1,12 +1,12 @@
``pxar`` is a command line utility for creating and manipulating archives in the ``pxar`` is a command line utility to create and manipulate archives in the
:ref:`pxar-format`. :ref:`pxar-format`.
It is inspired by `casync file archive format It is inspired by `casync file archive format
<http://0pointer.net/blog/casync-a-tool-for-distributing-file-system-images.html>`_, <http://0pointer.net/blog/casync-a-tool-for-distributing-file-system-images.html>`_,
which caters to a similar use-case. which caters to a similar use-case.
The ``.pxar`` format is adapted to fulfill the specific needs of the Proxmox The ``.pxar`` format is adapted to fulfill the specific needs of the Proxmox
Backup Server, for example, efficient storage of hard links. Backup Server, for example, efficient storage of hardlinks.
The format is designed to reduce the required storage on the server by The format is designed to reduce storage space needed on the server by achieving
achieving a high level of deduplication. a high level of deduplication.
Creating an Archive Creating an Archive
^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
@ -24,10 +24,10 @@ This will create a new archive called ``archive.pxar`` with the contents of the
the same name is already present in the target folder, the creation will the same name is already present in the target folder, the creation will
fail. fail.
By default, ``pxar`` will skip certain mount points and will not follow device By default, ``pxar`` will skip certain mountpoints and will not follow device
boundaries. This design decision is based on the primary use case of creating boundaries. This design decision is based on the primary use case of creating
archives for backups. It makes sense to ignore the contents of certain archives for backups. It makes sense to not back up the contents of certain
temporary or system specific files in a backup. temporary or system specific files.
To alter this behavior and follow device boundaries, use the To alter this behavior and follow device boundaries, use the
``--all-file-systems`` flag. ``--all-file-systems`` flag.
@ -41,38 +41,40 @@ by running:
# pxar create archive.pxar /path/to/source --exclude '**/*.txt' # pxar create archive.pxar /path/to/source --exclude '**/*.txt'
Be aware that the shell itself will try to expand glob patterns before invoking Be aware that the shell itself will try to expand all of the glob patterns before
``pxar``. In order to avoid this, all globs have to be quoted correctly. invoking ``pxar``.
In order to avoid this, all globs have to be quoted correctly.
It is possible to pass the ``--exclude`` parameter multiple times, in order to It is possible to pass the ``--exclude`` parameter multiple times, in order to
match more than one pattern. This allows you to use more complex match more than one pattern. This allows you to use more complex
file inclusion/exclusion behavior. However, it is recommended to use file exclusion/inclusion behavior. However, it is recommended to use
``.pxarexclude`` files instead for such cases. ``.pxarexclude`` files instead for such cases.
For example you might want to exclude all ``.txt`` files except a specific For example you might want to exclude all ``.txt`` files except for a specific
one from the archive. This would be achieved via the negated match pattern, one from the archive. This is achieved via the negated match pattern, prefixed
prefixed by ``!``. All the glob patterns are relative to the ``source`` by ``!``.
directory. All the glob patterns are relative to the ``source`` directory.
.. code-block:: console .. code-block:: console
# pxar create archive.pxar /path/to/source --exclude '**/*.txt' --exclude '!/folder/file.txt' # pxar create archive.pxar /path/to/source --exclude '**/*.txt' --exclude '!/folder/file.txt'
.. NOTE:: The order of the glob match patterns matters, as later ones override .. NOTE:: The order of the glob match patterns matters as later ones override
earlier ones. Permutations of the same patterns lead to different results. previous ones. Permutations of the same patterns lead to different results.
``pxar`` will store the list of glob match patterns passed as parameters via the ``pxar`` will store the list of glob match patterns passed as parameters via the
command line, in a file called ``.pxarexclude-cli``, at the root of the archive. command line, in a file called ``.pxarexclude-cli`` at the root of
the archive.
If a file with this name is already present in the source folder during archive If a file with this name is already present in the source folder during archive
creation, this file is not included in the archive, and the file containing the creation, this file is not included in the archive and the file containing the
new patterns is added to the archive instead. The original file is not altered. new patterns is added to the archive instead, the original file is not altered.
A more convenient and persistent way to exclude files from the archive is by A more convenient and persistent way to exclude files from the archive is by
placing the glob match patterns in ``.pxarexclude`` files. placing the glob match patterns in ``.pxarexclude`` files.
It is possible to create and place these files in any directory of the filesystem It is possible to create and place these files in any directory of the filesystem
tree. tree.
These files must contain one pattern per line, and later patterns override These files must contain one pattern per line, again later patterns win over
earlier ones. previous ones.
The patterns control file exclusions of files present within the given directory The patterns control file exclusions of files present within the given directory
or further below it in the tree. or further below it in the tree.
The behavior is the same as described in :ref:`client_creating_backups`. The behavior is the same as described in :ref:`client_creating_backups`.
@ -87,7 +89,7 @@ with the following command:
# pxar extract archive.pxar /path/to/target # pxar extract archive.pxar /path/to/target
If no target is provided, the contents of the archive is extracted to the current If no target is provided, the content of the archive is extracted to the current
working directory. working directory.
In order to restore only parts of an archive, single files, and/or folders, In order to restore only parts of an archive, single files, and/or folders,
@ -114,13 +116,13 @@ run the following command:
# pxar list archive.pxar # pxar list archive.pxar
This displays the full path of each file or directory with respect to the This displays the full path of each file or directory with respect to the
archive's root. archives root.
Mounting an Archive Mounting an Archive
^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
``pxar`` allows you to mount and inspect the contents of an archive via _`FUSE`. ``pxar`` allows you to mount and inspect the contents of an archive via _`FUSE`.
In order to mount an archive named ``archive.pxar`` to the mount point ``/mnt``, In order to mount an archive named ``archive.pxar`` to the mountpoint ``/mnt``,
run the command: run the command:
.. code-block:: console .. code-block:: console
@ -128,7 +130,7 @@ run the command:
# pxar mount archive.pxar /mnt # pxar mount archive.pxar /mnt
Once the archive is mounted, you can access its content under the given Once the archive is mounted, you can access its content under the given
mount point. mountpoint.
.. code-block:: console .. code-block:: console

View File

@ -11,16 +11,11 @@ Disk Management
:alt: List of disks :alt: List of disks
Proxmox Backup Server comes with a set of disk utilities, which are Proxmox Backup Server comes with a set of disk utilities, which are
accessed using the ``disk`` subcommand or the web interface. This subcommand accessed using the ``disk`` subcommand. This subcommand allows you to initialize
allows you to initialize disks, create various filesystems, and get information disks, create various filesystems, and get information about the disks.
about the disks.
.. image:: images/screenshots/pbs-gui-disks.png
:align: right
:alt: Web Interface Administration: Disks
To view the disks connected to the system, navigate to **Administration -> To view the disks connected to the system, navigate to **Administration ->
Storage/Disks** in the web interface or use the ``list`` subcommand of Disks** in the web interface or use the ``list`` subcommand of
``disk``: ``disk``:
.. code-block:: console .. code-block:: console
@ -47,9 +42,9 @@ To initialize a disk with a new GPT, use the ``initialize`` subcommand:
:alt: Create a directory :alt: Create a directory
You can create an ``ext4`` or ``xfs`` filesystem on a disk using ``fs You can create an ``ext4`` or ``xfs`` filesystem on a disk using ``fs
create``, or by navigating to **Administration -> Storage/Disks -> Directory** create``, or by navigating to **Administration -> Disks -> Directory** in the
in the web interface and creating one from there. The following command creates web interface and creating one from there. The following command creates an
an ``ext4`` filesystem and passes the ``--add-datastore`` parameter, in order to ``ext4`` filesystem and passes the ``--add-datastore`` parameter, in order to
automatically create a datastore on the disk (in this case ``sdd``). This will automatically create a datastore on the disk (in this case ``sdd``). This will
create a datastore at the location ``/mnt/datastore/store1``: create a datastore at the location ``/mnt/datastore/store1``:
@ -62,7 +57,7 @@ create a datastore at the location ``/mnt/datastore/store1``:
:alt: Create ZFS :alt: Create ZFS
You can also create a ``zpool`` with various raid levels from **Administration You can also create a ``zpool`` with various raid levels from **Administration
-> Storage/Disks -> ZFS** in the web interface, or by using ``zpool create``. The command -> Disks -> Zpool** in the web interface, or by using ``zpool create``. The command
below creates a mirrored ``zpool`` using two disks (``sdb`` & ``sdc``) and below creates a mirrored ``zpool`` using two disks (``sdb`` & ``sdc``) and
mounts it under ``/mnt/datastore/zpool1``: mounts it under ``/mnt/datastore/zpool1``:
@ -95,10 +90,6 @@ display S.M.A.R.T. attributes from the web interface or by using the command:
:term:`Datastore` :term:`Datastore`
----------------- -----------------
.. image:: images/screenshots/pbs-gui-datastore-summary.png
:align: right
:alt: Datastore Usage Overview
A datastore refers to a location at which backups are stored. The current A datastore refers to a location at which backups are stored. The current
implementation uses a directory inside a standard Unix file system (``ext4``, implementation uses a directory inside a standard Unix file system (``ext4``,
``xfs`` or ``zfs``) to store the backup data. ``xfs`` or ``zfs``) to store the backup data.
@ -111,7 +102,7 @@ is stored in the file ``/etc/proxmox-backup/datastore.cfg``.
subdirectories per directory. That number comes from the 2\ :sup:`16` subdirectories per directory. That number comes from the 2\ :sup:`16`
pre-created chunk namespace directories, and the ``.`` and ``..`` default pre-created chunk namespace directories, and the ``.`` and ``..`` default
directory entries. This requirement excludes certain filesystems and directory entries. This requirement excludes certain filesystems and
filesystem configurations from being supported for a datastore. For example, filesystem configuration from being supported for a datastore. For example,
``ext3`` as a whole or ``ext4`` with the ``dir_nlink`` feature manually disabled. ``ext3`` as a whole or ``ext4`` with the ``dir_nlink`` feature manually disabled.
@ -120,24 +111,23 @@ Datastore Configuration
.. image:: images/screenshots/pbs-gui-datastore-content.png .. image:: images/screenshots/pbs-gui-datastore-content.png
:align: right :align: right
:alt: Datastore Content Overview :alt: Datastore Overview
You can configure multiple datastores. A minimum of one datastore needs to be You can configure multiple datastores. Minimum one datastore needs to be
configured. The datastore is identified by a simple *name* and points to a configured. The datastore is identified by a simple *name* and points to a
directory on the filesystem. Each datastore also has associated retention directory on the filesystem. Each datastore also has associated retention
settings of how many backup snapshots for each interval of ``hourly``, settings of how many backup snapshots for each interval of ``hourly``,
``daily``, ``weekly``, ``monthly``, ``yearly`` as well as a time-independent ``daily``, ``weekly``, ``monthly``, ``yearly`` as well as a time-independent
number of backups to keep in that store. :ref:`backup-pruning` and number of backups to keep in that store. :ref:`backup-pruning` and
:ref:`garbage collection <client_garbage-collection>` can also be configured to :ref:`garbage collection <client_garbage-collection>` can also be configured to run
run periodically, based on a configured schedule (see periodically based on a configured schedule (see :ref:`calendar-event-scheduling`) per datastore.
:ref:`calendar-event-scheduling`) per datastore.
.. _storage_datastore_create: .. _storage_datastore_create:
Creating a Datastore Creating a Datastore
^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^
.. image:: images/screenshots/pbs-gui-datastore-create.png .. image:: images/screenshots/pbs-gui-datastore-create-general.png
:align: right :align: right
:alt: Create a datastore :alt: Create a datastore
@ -156,8 +146,7 @@ window:
* *Comment* can be used to add some contextual information to the datastore. * *Comment* can be used to add some contextual information to the datastore.
Alternatively you can create a new datastore from the command line. The Alternatively you can create a new datastore from the command line. The
following command creates a new datastore called ``store1`` on following command creates a new datastore called ``store1`` on :file:`/backup/disk1/store1`
:file:`/backup/disk1/store1`
.. code-block:: console .. code-block:: console
@ -167,7 +156,7 @@ following command creates a new datastore called ``store1`` on
Managing Datastores Managing Datastores
^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
To list existing datastores from the command line, run: To list existing datastores from the command line run:
.. code-block:: console .. code-block:: console
@ -227,9 +216,8 @@ After creating a datastore, the following default layout will appear:
`.lock` is an empty file used for process locking. `.lock` is an empty file used for process locking.
The `.chunks` directory contains folders, starting from `0000` and increasing in The `.chunks` directory contains folders, starting from `0000` and taking hexadecimal values until `ffff`. These
hexadecimal values until `ffff`. These directories will store the chunked data, directories will store the chunked data after a backup operation has been executed.
categorized by checksum, after a backup operation has been executed.
.. code-block:: console .. code-block:: console
@ -261,57 +249,3 @@ categorized by checksum, after a backup operation has been executed.
276490 drwxr-x--- 1 backup backup 1.1M Jul 8 12:35 . 276490 drwxr-x--- 1 backup backup 1.1M Jul 8 12:35 .
Once you uploaded some backups, or created namespaces, you may see the Backup
Type (`ct`, `vm`, `host`) and the start of the namespace hierarchy (`ns`).
.. _storage_namespaces:
Backup Namespaces
~~~~~~~~~~~~~~~~~
A datastore can host many backups as long as the underlying storage is big
enough and provides the performance required for one's use case.
But, without any hierarchy or separation its easy to run into naming conflicts,
especially when using the same datastore for multiple Proxmox VE instances or
multiple users.
The backup namespace hierarchy allows you to clearly separate different users
or backup sources in general, avoiding naming conflicts and providing
well-organized backup content view.
Each namespace level can host any backup type, CT, VM or Host but also other
namespaces, up to a depth of 8 level, where the root namespace is the first
level.
Namespace Permissions
^^^^^^^^^^^^^^^^^^^^^
You can make the permission configuration of a datastore more fine-grained by
setting permissions only on a specific namespace.
To see a datastore you need permission that has at least one of `AUDIT`,
`MODIFY`, `READ` or `BACKUP` privilege on any namespace it contains.
To create or delete a namespace you require the modify privilege on the parent
namespace. So, to initially create namespaces you need to have a permission
with a access role that includes the `MODIFY` privilege on the datastore itself.
For backup groups the existing privilege rules still apply, you either need a
powerful permission or be the owner of the backup group, nothing changed here.
.. todo:: continue
Options
~~~~~~~
.. image:: images/screenshots/pbs-gui-datastore-options.png
:align: right
:alt: Datastore Options
There are a few per-datastore options:
* :ref:`Notifications <maintenance_notification>`
* :ref:`Maintenance Mode <maintenance_mode>`
* Verification of incoming backups

View File

@ -4,8 +4,8 @@ Host System Administration
========================== ==========================
`Proxmox Backup`_ is based on the famous Debian_ Linux `Proxmox Backup`_ is based on the famous Debian_ Linux
distribution. This means that you have access to the entire range of distribution. That means that you have access to the whole world of
Debian packages, and that the base system is well documented. The `Debian Debian packages, and the base system is well documented. The `Debian
Administrator's Handbook`_ is available online, and provides a Administrator's Handbook`_ is available online, and provides a
comprehensive introduction to the Debian operating system. comprehensive introduction to the Debian operating system.
@ -15,21 +15,17 @@ through that channel. In addition, we provide our own package
repository to roll out all Proxmox related packages. This includes repository to roll out all Proxmox related packages. This includes
updates to some Debian packages when necessary. updates to some Debian packages when necessary.
We also deliver a specially optimized Linux kernel, based on the Ubuntu We also deliver a specially optimized Linux kernel, where we enable
kernel. That kernel includes drivers for ZFS_. all required virtualization and container features. That kernel
includes drivers for ZFS_, and several hardware drivers. For example,
we ship Intel network card drivers to support their newest hardware.
The following sections will concentrate on backup related topics. They The following sections will concentrate on backup related topics. They
will explain things which are different on `Proxmox Backup`_, or either explain things which are different on `Proxmox Backup`_, or
tasks which are commonly used on `Proxmox Backup`_. For other topics, tasks which are commonly used on `Proxmox Backup`_. For other topics,
please refer to the standard Debian documentation. please refer to the standard Debian documentation.
.. include:: local-zfs.rst .. include:: local-zfs.rst
.. include:: system-booting.rst
.. include:: certificate-management.rst
.. include:: services.rst .. include:: services.rst
.. include:: command-line-tools.rst

View File

@ -1,379 +0,0 @@
.. _chapter-systembooting:
Host Bootloader
---------------
`Proxmox Backup`_ currently uses one of two bootloaders depending on the disk setup
selected in the installer.
For EFI Systems installed with ZFS as the root filesystem ``systemd-boot`` is
used. All other deployments use the standard ``grub`` bootloader (this usually
also applies to systems which are installed on top of Debian).
.. _systembooting-installer-part-scheme:
Partitioning Scheme Used by the Installer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The `Proxmox Backup`_ installer creates 3 partitions on all disks selected for
installation.
The created partitions are:
* a 1 MB BIOS Boot Partition (gdisk type EF02)
* a 512 MB EFI System Partition (ESP, gdisk type EF00)
* a third partition spanning the set ``hdsize`` parameter or the remaining space
used for the chosen storage type
Systems using ZFS as root filesystem are booted with a kernel and initrd image
stored on the 512 MB EFI System Partition. For legacy BIOS systems, ``grub`` is
used, for EFI systems ``systemd-boot`` is used. Both are installed and configured
to point to the ESPs.
``grub`` in BIOS mode (``--target i386-pc``) is installed onto the BIOS Boot
Partition of all selected disks on all systems booted with ``grub`` (These are
all installs with root on ``ext4`` or ``xfs`` and installs with root on ZFS on
non-EFI systems).
.. _systembooting-proxmox-boot-tool:
Synchronizing the content of the ESP with ``proxmox-boot-tool``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``proxmox-boot-tool`` is a utility used to keep the contents of the EFI System
Partitions properly configured and synchronized. It copies certain kernel
versions to all ESPs and configures the respective bootloader to boot from
the ``vfat`` formatted ESPs. In the context of ZFS as root filesystem this means
that you can use all optional features on your root pool instead of the subset
which is also present in the ZFS implementation in ``grub`` or having to create a
separate small boot-pool (see: `Booting ZFS on root with grub
<https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS>`_).
In setups with redundancy all disks are partitioned with an ESP, by the
installer. This ensures the system boots even if the first boot device fails
or if the BIOS can only boot from a particular disk.
The ESPs are not kept mounted during regular operation. This helps to prevent
filesystem corruption to the ``vfat`` formatted ESPs in case of a system crash,
and removes the need to manually adapt ``/etc/fstab`` in case the primary boot
device fails.
``proxmox-boot-tool`` handles the following tasks:
* formatting and setting up a new partition
* copying and configuring new kernel images and initrd images to all listed ESPs
* synchronizing the configuration on kernel upgrades and other maintenance tasks
* managing the list of kernel versions which are synchronized
* configuring the boot-loader to boot a particular kernel version (pinning)
You can view the currently configured ESPs and their state by running:
.. code-block:: console
# proxmox-boot-tool status
.. _systembooting-proxmox-boot-setup:
Setting up a new partition for use as synced ESP
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To format and initialize a partition as synced ESP, e.g., after replacing a
failed vdev in an rpool, ``proxmox-boot-tool`` from ``pve-kernel-helper`` can be used.
WARNING: the ``format`` command will format the ``<partition>``, make sure to pass
in the right device/partition!
For example, to format an empty partition ``/dev/sda2`` as ESP, run the following:
.. code-block:: console
# proxmox-boot-tool format /dev/sda2
To setup an existing, unmounted ESP located on ``/dev/sda2`` for inclusion in
`Proxmox Backup`_'s kernel update synchronization mechanism, use the following:
.. code-block:: console
# proxmox-boot-tool init /dev/sda2
Afterwards `/etc/kernel/proxmox-boot-uuids`` should contain a new line with the
UUID of the newly added partition. The ``init`` command will also automatically
trigger a refresh of all configured ESPs.
.. _systembooting-proxmox-boot-refresh:
Updating the configuration on all ESPs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To copy and configure all bootable kernels and keep all ESPs listed in
``/etc/kernel/proxmox-boot-uuids`` in sync you just need to run:
.. code-block:: console
# proxmox-boot-tool refresh
(The equivalent to running ``update-grub`` systems with ``ext4`` or ``xfs`` on root).
This is necessary should you make changes to the kernel commandline, or want to
sync all kernels and initrds.
.. NOTE:: Both ``update-initramfs`` and ``apt`` (when necessary) will automatically
trigger a refresh.
Kernel Versions considered by ``proxmox-boot-tool``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The following kernel versions are configured by default:
* the currently running kernel
* the version being newly installed on package updates
* the two latest already installed kernels
* the latest version of the second-to-last kernel series (e.g. 5.0, 5.3), if applicable
* any manually selected kernels
Manually keeping a kernel bootable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Should you wish to add a certain kernel and initrd image to the list of
bootable kernels use ``proxmox-boot-tool kernel add``.
For example run the following to add the kernel with ABI version ``5.0.15-1-pve``
to the list of kernels to keep installed and synced to all ESPs:
.. code-block:: console
# proxmox-boot-tool kernel add 5.0.15-1-pve
``proxmox-boot-tool kernel list`` will list all kernel versions currently selected
for booting:
.. code-block:: console
# proxmox-boot-tool kernel list
Manually selected kernels:
5.0.15-1-pve
Automatically selected kernels:
5.0.12-1-pve
4.15.18-18-pve
Run ``proxmox-boot-tool kernel remove`` to remove a kernel from the list of
manually selected kernels, for example:
.. code-block:: console
# proxmox-boot-tool kernel remove 5.0.15-1-pve
.. NOTE:: It's required to run ``proxmox-boot-tool refresh`` to update all EFI System
Partitions (ESPs) after a manual kernel addition or removal from above.
.. _systembooting-determine-bootloader:
Determine which Bootloader is Used
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. image:: images/screenshots/boot-grub.png
:target: _images/boot-grub.png
:align: left
:alt: Grub boot screen
The simplest and most reliable way to determine which bootloader is used, is to
watch the boot process of the `Proxmox Backup`_ node.
You will either see the blue box of ``grub`` or the simple black on white
``systemd-boot``.
.. image:: images/screenshots/boot-systemdboot.png
:target: _images/boot-systemdboot.png
:align: right
:alt: systemd-boot screen
Determining the bootloader from a running system might not be 100% accurate. The
safest way is to run the following command:
.. code-block:: console
# efibootmgr -v
If it returns a message that EFI variables are not supported, ``grub`` is used in
BIOS/Legacy mode.
If the output contains a line that looks similar to the following, ``grub`` is
used in UEFI mode.
.. code-block:: console
Boot0005* proxmox [...] File(\EFI\proxmox\grubx64.efi)
If the output contains a line similar to the following, ``systemd-boot`` is used.
.. code-block:: console
Boot0006* Linux Boot Manager [...] File(\EFI\systemd\systemd-bootx64.efi)
By running:
.. code-block:: console
# proxmox-boot-tool status
you can find out if ``proxmox-boot-tool`` is configured, which is a good
indication of how the system is booted.
.. _systembooting-grub:
Grub
~~~~
``grub`` has been the de-facto standard for booting Linux systems for many years
and is quite well documented
(see the `Grub Manual
<https://www.gnu.org/software/grub/manual/grub/grub.html>`_).
Configuration
^^^^^^^^^^^^^
Changes to the ``grub`` configuration are done via the defaults file
``/etc/default/grub`` or config snippets in ``/etc/default/grub.d``. To regenerate
the configuration file after a change to the configuration run:
.. code-block:: console
# update-grub
.. NOTE:: Systems using ``proxmox-boot-tool`` will call
``proxmox-boot-tool refresh`` upon ``update-grub``
.. _systembooting-systemdboot:
Systemd-boot
~~~~~~~~~~~~
``systemd-boot`` is a lightweight EFI bootloader. It reads the kernel and initrd
images directly from the EFI Service Partition (ESP) where it is installed.
The main advantage of directly loading the kernel from the ESP is that it does
not need to reimplement the drivers for accessing the storage. In `Proxmox
Backup`_ :ref:`proxmox-boot-tool <systembooting-proxmox-boot-tool>` is used to
keep the configuration on the ESPs synchronized.
.. _systembooting-systemd-boot-config:
Configuration
^^^^^^^^^^^^^
``systemd-boot`` is configured via the file ``loader/loader.conf`` in the root
directory of an EFI System Partition (ESP). See the ``loader.conf(5)`` manpage
for details.
Each bootloader entry is placed in a file of its own in the directory
``loader/entries/``
An example entry.conf looks like this (``/`` refers to the root of the ESP):
.. code-block:: console
title Proxmox
version 5.0.15-1-pve
options root=ZFS=rpool/ROOT/pve-1 boot=zfs
linux /EFI/proxmox/5.0.15-1-pve/vmlinuz-5.0.15-1-pve
initrd /EFI/proxmox/5.0.15-1-pve/initrd.img-5.0.15-1-pve
.. _systembooting-edit-kernel-cmdline:
Editing the Kernel Commandline
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can modify the kernel commandline in the following places, depending on the
bootloader used:
Grub
^^^^
The kernel commandline needs to be placed in the variable
``GRUB_CMDLINE_LINUX_DEFAULT`` in the file ``/etc/default/grub``. Running
``update-grub`` appends its content to all ``linux`` entries in
``/boot/grub/grub.cfg``.
Systemd-boot
^^^^^^^^^^^^
The kernel commandline needs to be placed as one line in ``/etc/kernel/cmdline``.
To apply your changes, run ``proxmox-boot-tool refresh``, which sets it as the
``option`` line for all config files in ``loader/entries/proxmox-*.conf``.
.. _systembooting-kernel-pin:
Override the Kernel-Version for next Boot
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To select a kernel that is not currently the default kernel, you can either:
* use the boot loader menu that is displayed at the beginning of the boot
process
* use the ``proxmox-boot-tool`` to ``pin`` the system to a kernel version either
once or permanently (until pin is reset).
This should help you work around incompatibilities between a newer kernel
version and the hardware.
.. NOTE:: Such a pin should be removed as soon as possible so that all current
security patches of the latest kernel are also applied to the system.
For example: To permanently select the version ``5.15.30-1-pve`` for booting you
would run:
.. code-block:: console
# proxmox-boot-tool kernel pin 5.15.30-1-pve
.. TIP:: The pinning functionality works for all `Proxmox Backup`_ systems, not only those using
``proxmox-boot-tool`` to synchronize the contents of the ESPs, if your system
does not use ``proxmox-boot-tool`` for synchronizing you can also skip the
``proxmox-boot-tool refresh`` call in the end.
You can also set a kernel version to be booted on the next system boot only.
This is for example useful to test if an updated kernel has resolved an issue,
which caused you to ``pin`` a version in the first place:
.. code-block:: console
# proxmox-boot-tool kernel pin 5.15.30-1-pve --next-boot
To remove any pinned version configuration use the ``unpin`` subcommand:
.. code-block:: console
# proxmox-boot-tool kernel unpin
While ``unpin`` has a ``--next-boot`` option as well, it is used to clear a pinned
version set with ``--next-boot``. As that happens already automatically on boot,
invonking it manually is of little use.
After setting, or clearing pinned versions you also need to synchronize the
content and configuration on the ESPs by running the ``refresh`` subcommand.
.. TIP:: You will be prompted to automatically do for ``proxmox-boot-tool`` managed
systems if you call the tool interactively.
.. code-block:: console
# proxmox-boot-tool refresh

View File

@ -3,6 +3,9 @@
Tape Backup Tape Backup
=========== ===========
.. CAUTION:: Tape Backup is a technical preview feature, not meant for
production use.
.. image:: images/screenshots/pbs-gui-tape-changer-overview.png .. image:: images/screenshots/pbs-gui-tape-changer-overview.png
:align: right :align: right
:alt: Tape Backup: Tape changer overview :alt: Tape Backup: Tape changer overview
@ -500,7 +503,7 @@ a single media pool, so a job only uses tapes from that pool.
is less space efficient, because the media from the last set is less space efficient, because the media from the last set
may not be fully written, leaving the remaining space unused. may not be fully written, leaving the remaining space unused.
The advantage is that this produces media sets of minimal The advantage is that this procudes media sets of minimal
size. Small sets are easier to handle, can be moved more conveniently size. Small sets are easier to handle, can be moved more conveniently
to an off-site vault, and can be restored much faster. to an off-site vault, and can be restored much faster.
@ -519,9 +522,8 @@ a single media pool, so a job only uses tapes from that pool.
This balances between space efficiency and media count. This balances between space efficiency and media count.
.. NOTE:: Retention period starts on the creation time of the next .. NOTE:: Retention period starts when the calendar event
media-set or, if that does not exist, when the calendar event triggers.
triggers the next time after the current media-set start time.
Additionally, the following events may allocate a new media set: Additionally, the following events may allocate a new media set:
@ -565,6 +567,13 @@ a single media pool, so a job only uses tapes from that pool.
the password. Please make sure to remember the password, in case the password. Please make sure to remember the password, in case
you need to restore the key. you need to restore the key.
.. NOTE:: We use global content namespace, meaning we do not store the
source datastore name. Because of this, it is impossible to distinguish
store1:/vm/100 from store2:/vm/100. Please use different media pools
if the sources are from different namespaces with conflicting names
(for example, if the sources are from different Proxmox VE clusters).
.. image:: images/screenshots/pbs-gui-tape-pools-add.png .. image:: images/screenshots/pbs-gui-tape-pools-add.png
:align: right :align: right
:alt: Tape Backup: Add a media pool :alt: Tape Backup: Add a media pool
@ -681,16 +690,6 @@ To remove a job, please use:
# proxmox-tape backup-job remove job2 # proxmox-tape backup-job remove job2
By default, all (recursive) namespaces of the datastore are included in a tape
backup. You can specify a single namespace with ``ns`` and a depth with
``max-depth``. For example:
.. code-block:: console
# proxmox-tape backup-job update job2 --ns mynamespace --max-depth 3
If no `max-depth` is given, it will include all recursive namespaces.
.. image:: images/screenshots/pbs-gui-tape-backup-jobs-add.png .. image:: images/screenshots/pbs-gui-tape-backup-jobs-add.png
:align: right :align: right
:alt: Tape Backup: Add a backup job :alt: Tape Backup: Add a backup job
@ -807,16 +806,6 @@ The following options are available:
media set into import-export slots. The operator can then pick up media set into import-export slots. The operator can then pick up
those tapes and move them to a media vault. those tapes and move them to a media vault.
--ns The namespace to backup.
If you only want to backup a specific namespace. If omitted, the root
namespaces is assumed.
--max-depth The depth to recurse namespaces.
``0`` means no recursion at all (only the given namespace). If omitted,
all namespaces are recursed (below the the given one).
Restore from Tape Restore from Tape
~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~
@ -851,53 +840,6 @@ data disk (datastore):
# proxmox-tape restore 9da37a55-aac7-4deb-91c6-482b3b675f30 mystore # proxmox-tape restore 9da37a55-aac7-4deb-91c6-482b3b675f30 mystore
Single Snapshot Restore
^^^^^^^^^^^^^^^^^^^^^^^
Sometimes it is not necessary to restore a whole media-set, but only some
specific snapshots from the tape. This can be achieved with the ``snapshots``
parameter:
.. code-block:: console
// proxmox-tape restore <media-set-uuid> <datastore> [<snapshot>]
# proxmox-tape restore 9da37a55-aac7-4deb-91c6-482b3b675f30 mystore sourcestore:host/hostname/2022-01-01T00:01:00Z
This first restores the snapshot to a temporary location, then restores the relevant
chunk archives, and finally restores the snapshot data to the target datastore.
The ``snapshot`` parameter can be given multiple times, so one can restore
multiple snapshots with one restore action.
.. NOTE:: When using the single snapshot restore, the tape must be traversed
more than once, which, if you restore many snapshots at once, can take longer
than restoring the whole datastore.
Namespaces
^^^^^^^^^^
It is also possible to select and map specific namespaces from a media-set
during a restore. This is possible with the ``namespaces`` parameter.
The format of the parameter is
.. code-block:: console
store=<source-datastore>[,source=<source-ns>][,target=<target-ns>][,max-depth=<depth>]
If ``source`` or ``target`` is not given, the root namespace is assumed.
When no ``max-depth`` is given, the source namespace will be fully recursed.
An example restore command:
.. code-block:: console
# proxmox-tape restore 9da37a55-aac7-4deb-91c6-482b3b675f30 mystore --namespaces store=sourcedatastore,source=ns1,target=ns2,max-depth=2
The parameter can be given multiple times. It can also be combined with the
``snapshots`` parameter to only restore those snapshots and map them to different
namespaces.
Update Inventory Update Inventory
~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~
@ -906,17 +848,6 @@ Update Inventory
Restore Catalog Restore Catalog
~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~
To restore a catalog from an existing tape, just insert the tape into the drive
and execute:
.. code-block:: console
# proxmox-tape catalog
You can restore from a tape even without an existing catalog, but only the
whole media set. If you do this, the catalog will be automatically created.
Encryption Key Management Encryption Key Management
~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
@ -1039,76 +970,3 @@ This command does the following:
- run drive cleaning operation - run drive cleaning operation
- unload the cleaning tape (to slot 3) - unload the cleaning tape (to slot 3)
Example Setups
--------------
Here are a few example setups for how to manage media pools and schedules.
This is not an exhaustive list, and there are many more possible combinations
of useful settings.
Single Continued Media Set
~~~~~~~~~~~~~~~~~~~~~~~~~~
The most simple setup: always continue the media-set and never expire.
Allocation policy:
continue
Retention policy:
keep
This setup has the advantage of being easy to manage and is re-using the benefits
from deduplication as much as possible. But, it's also prone to a failure of
any single tape, which would render all backups referring to chunks from that
tape unusable.
If you want to start a new media-set manually, you can set the currently
writable media of the set either to 'full', or set the location to an
offsite vault.
Weekday Scheme
~~~~~~~~~~~~~~
A slightly more complex scheme, where the goal is to have an independent
tape or media set for each weekday, for example from Monday to Friday.
This can be solved by having a separate media pool for each day, so 'Monday',
'Tuesday', etc.
Allocation policy:
should be 'mon' for the 'Monday' pool, 'tue' for the Tuesday pool and so on.
Retention policy:
overwrite
There should be a (or more) tape-backup jobs for each pool on the corresponding
weekday. This scheme is still very manageable with one media set per weekday,
and could be easily moved off-site.
Multiple Pools with Different Policies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Complex setups are also possible with multiple media pools configured with
different allocation and retention policies.
An example would be to have two media pools. The first configured with weekly
allocation and a few weeks of retention:
Allocation policy:
mon
Retention policy:
3 weeks
The second pool configured yearly allocation that does not expire:
Allocation policy:
yearly
Retention policy:
keep
In combination with suited prune settings and tape backup schedules, this
achieves long-term storage of some backups, while keeping the current
backups on smaller media sets that get expired every three plus the current
week (~ 4 weeks).

View File

@ -8,7 +8,7 @@ Datastores
A Datastore is the logical place where :ref:`Backup Snapshots A Datastore is the logical place where :ref:`Backup Snapshots
<term_backup_snapshot>` and their chunks are stored. Snapshots consist of a <term_backup_snapshot>` and their chunks are stored. Snapshots consist of a
manifest, blobs, and dynamic- and fixed-indexes (see :ref:`terms`), and are manifest, blobs, dynamic- and fixed-indexes (see :ref:`terms`), and are
stored in the following directory structure: stored in the following directory structure:
<datastore-root>/<type>/<id>/<time>/ <datastore-root>/<type>/<id>/<time>/
@ -32,8 +32,8 @@ The chunks of a datastore are found in
<datastore-root>/.chunks/ <datastore-root>/.chunks/
This chunk directory is further subdivided by the first four bytes of the This chunk directory is further subdivided by the first four byte of the chunks
chunk's checksum, so a chunk with the checksum checksum, so the chunk with the checksum
a342e8151cbf439ce65f3df696b54c67a114982cc0aa751f2852c2f7acc19a8b a342e8151cbf439ce65f3df696b54c67a114982cc0aa751f2852c2f7acc19a8b
@ -47,7 +47,7 @@ per directory can be bad for file system performance.
These chunk directories ('0000'-'ffff') will be preallocated when a datastore These chunk directories ('0000'-'ffff') will be preallocated when a datastore
is created. is created.
Fixed-Sized Chunks Fixed-sized Chunks
^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^
For block based backups (like VMs), fixed-sized chunks are used. The content For block based backups (like VMs), fixed-sized chunks are used. The content
@ -58,10 +58,10 @@ often tries to allocate files in contiguous pieces, so new files get new
blocks, and changing existing files changes only their own blocks. blocks, and changing existing files changes only their own blocks.
As an optimization, VMs in `Proxmox VE`_ can make use of 'dirty bitmaps', which As an optimization, VMs in `Proxmox VE`_ can make use of 'dirty bitmaps', which
can track the changed blocks of an image. Since these bitmaps are also a can track the changed blocks of an image. Since these bitmap are also a
representation of the image split into chunks, there is a direct relation representation of the image split into chunks, there is a direct relation
between the dirty blocks of the image and chunks which need to be uploaded. between dirty blocks of the image and chunks which need to get uploaded, so
Thus, only modified chunks of the disk need to be uploaded to a backup. only modified chunks of the disk have to be uploaded for a backup.
Since the image is always split into chunks of the same size, unchanged blocks Since the image is always split into chunks of the same size, unchanged blocks
will result in identical checksums for those chunks, so such chunks do not need will result in identical checksums for those chunks, so such chunks do not need
@ -71,24 +71,24 @@ changed blocks.
For consistency, `Proxmox VE`_ uses a QEMU internal snapshot mechanism, that For consistency, `Proxmox VE`_ uses a QEMU internal snapshot mechanism, that
does not rely on storage snapshots either. does not rely on storage snapshots either.
Dynamically Sized Chunks Dynamically sized Chunks
^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^
When working with file-based systems rather than block-based systems, If one does not want to backup block-based systems but rather file-based
using fixed-sized chunks is not a good idea, since every time a file systems, using fixed-sized chunks is not a good idea, since every time a file
would change in size, the remaining data would be shifted around, would change in size, the remaining data gets shifted around and this would
resulting in many chunks changing and the amount of deduplication being reduced. result in many chunks changing, reducing the amount of deduplication.
To improve this, `Proxmox Backup`_ Server uses dynamically sized chunks To improve this, `Proxmox Backup`_ Server uses dynamically sized chunks
instead. Instead of splitting an image into fixed sizes, it first generates a instead. Instead of splitting an image into fixed sizes, it first generates a
consistent file archive (:ref:`pxar <pxar-format>`) and uses a rolling hash consistent file archive (:ref:`pxar <pxar-format>`) and uses a rolling hash
over this on-the-fly generated archive to calculate chunk boundaries. over this on-the-fly generated archive to calculate chunk boundaries.
We use a variant of Buzhash which is a cyclic polynomial algorithm. It works We use a variant of Buzhash which is a cyclic polynomial algorithm. It works
by continuously calculating a checksum while iterating over the data, and on by continuously calculating a checksum while iterating over the data, and on
certain conditions, it triggers a hash boundary. certain conditions it triggers a hash boundary.
Assuming that most files on the system that is to be backed up have not Assuming that most files of the system that is to be backed up have not
changed, eventually the algorithm triggers the boundary on the same data as a changed, eventually the algorithm triggers the boundary on the same data as a
previous backup, resulting in chunks that can be reused. previous backup, resulting in chunks that can be reused.
@ -100,8 +100,8 @@ can be encrypted, and they are handled in a slightly different manner than
normal chunks. normal chunks.
The hashes of encrypted chunks are calculated not with the actual (encrypted) The hashes of encrypted chunks are calculated not with the actual (encrypted)
chunk content, but with the plain-text content, concatenated with the encryption chunk content, but with the plain-text content concatenated with the encryption
key. This way, two chunks with the same data but encrypted with different keys key. This way, two chunks of the same data encrypted with different keys
generate two different checksums and no collisions occur for multiple generate two different checksums and no collisions occur for multiple
encryption keys. encryption keys.
@ -112,14 +112,14 @@ the previous backup, do not need to be encrypted and uploaded.
Caveats and Limitations Caveats and Limitations
----------------------- -----------------------
Notes on Hash Collisions Notes on hash collisions
^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^
Every hashing algorithm has a chance to produce collisions, meaning two (or Every hashing algorithm has a chance to produce collisions, meaning two (or
more) inputs generate the same checksum. For SHA-256, this chance is more) inputs generate the same checksum. For SHA-256, this chance is
negligible. To calculate the chances of such a collision, one can use the ideas negligible. To calculate such a collision, one can use the ideas of the
of the 'birthday problem' from probability theory. For big numbers, this is 'birthday problem' from probability theory. For big numbers, this is actually
actually unfeasible to calculate with regular computers, but there is a good infeasible to calculate with regular computers, but there is a good
approximation: approximation:
.. math:: .. math::
@ -127,7 +127,7 @@ approximation:
p(n, d) = 1 - e^{-n^2/(2d)} p(n, d) = 1 - e^{-n^2/(2d)}
Where `n` is the number of tries, and `d` is the number of possibilities. Where `n` is the number of tries, and `d` is the number of possibilities.
For a concrete example, lets assume a large datastore of 1 PiB and an average For a concrete example lets assume a large datastore of 1 PiB, and an average
chunk size of 4 MiB. That means :math:`n = 268435456` tries, and :math:`d = chunk size of 4 MiB. That means :math:`n = 268435456` tries, and :math:`d =
2^{256}` possibilities. Inserting those values in the formula from earlier you 2^{256}` possibilities. Inserting those values in the formula from earlier you
will see that the probability of a collision in that scenario is: will see that the probability of a collision in that scenario is:
@ -136,96 +136,31 @@ will see that the probability of a collision in that scenario is:
3.1115 * 10^{-61} 3.1115 * 10^{-61}
For context, in a lottery game of guessing 6 numbers out of 45, the chance to For context, in a lottery game of guessing 6 out of 45, the chance to correctly
correctly guess all 6 numbers is only :math:`1.2277 * 10^{-7}`. This means the guess all 6 numbers is only :math:`1.2277 * 10^{-7}`, that means the chance of
chance of a collision is about the same as winning 13 such lottery games *in a a collision is about the same as winning 13 such lotto games *in a row*.
row*.
In conclusion, it is extremely unlikely that such a collision would occur by In conclusion, it is extremely unlikely that such a collision would occur by
accident in a normal datastore. accident in a normal datastore.
Additionally, SHA-256 is prone to length extension attacks, but since there is Additionally, SHA-256 is prone to length extension attacks, but since there is
an upper limit for how big the chunks are, this is not a problem, because a an upper limit for how big the chunk are, this is not a problem, since a
potential attacker cannot arbitrarily add content to the data beyond that potential attacker cannot arbitrarily add content to the data beyond that
limit. limit.
File-Based Backup File-based Backup
^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^
Since dynamically sized chunks (for file-based backups) are created on a custom Since dynamically sized chunks (for file-based backups) are created on a custom
archive format (pxar) and not over the files directly, there is no relation archive format (pxar) and not over the files directly, there is no relation
between the files and chunks. This means that the Proxmox Backup Client has to between files and the chunks. This means that the Proxmox Backup client has to
read all files again for every backup, otherwise it would not be possible to read all files again for every backup, otherwise it would not be possible to
generate a consistent, independent pxar archive where the original chunks can be generate a consistent independent pxar archive where the original chunks can be
reused. Note that in spite of this, only new or changed chunks will be uploaded. reused. Note that there will be still only new or change chunks be uploaded.
Verification of Encrypted Chunks Verification of encrypted chunks
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For encrypted chunks, only the checksum of the original (plaintext) data is For encrypted chunks, only the checksum of the original (plaintext) data is
available, making it impossible for the server (without the encryption key) to available, making it impossible for the server (without the encryption key), to
verify its content against it. Instead only the CRC-32 checksum gets checked. verify its content against it. Instead only the CRC-32 checksum gets checked.
Troubleshooting
---------------
Index files(*.fidx*, *.didx*) contain information about how to rebuild a file.
More precisely, they contain an ordered list of references to the chunks that
the original file was split into. If there is something wrong with a snapshot,
it might be useful to find out which chunks are referenced in it, and check
whether they are present and intact. The ``proxmox-backup-debug`` command line
tool can be used to inspect such files and recover their contents. For example,
to get a list of the referenced chunks of a *.fidx* index:
.. code-block:: console
# proxmox-backup-debug inspect file drive-scsi0.img.fidx
The same command can be used to inspect *.blob* files. Without the ``--decode``
parameter, just the size and the encryption type, if any, are printed. If
``--decode`` is set, the blob file is decoded into the specified file ('-' will
decode it directly to stdout).
The following example would print the decoded contents of
`qemu-server.conf.blob`. If the file you're trying to inspect is encrypted, a
path to the key file must be provided using ``--keyfile``.
.. code-block:: console
# proxmox-backup-debug inspect file qemu-server.conf.blob --decode -
You can also check in which index files a specific chunk file is referenced
with:
.. code-block:: console
# proxmox-backup-debug inspect chunk b531d3ffc9bd7c65748a61198c060678326a431db7eded874c327b7986e595e0 --reference-filter /path/in/a/datastore/directory
Here ``--reference-filter`` specifies where index files should be searched. This
can be an arbitrary path. If, for some reason, the filename of the chunk was
changed, you can explicitly specify the digest using ``--digest``. By default, the
chunk filename is used as the digest to look for. If no ``--reference-filter``
is specified, it will only print the CRC and encryption status of the chunk. You
can also decode chunks, by setting the ``--decode`` flag. If the chunk is
encrypted, a ``--keyfile`` must be provided, in order to decode it.
Restore without a Running Proxmox Backup Server
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
It's possible to restore specific files from a snapshot, without a running
Proxmox Backup Server instance, using the ``recover`` subcommand, provided you
have access to the intact index and chunk files. Note that you also need the
corresponding key file if the backup was encrypted.
.. code-block:: console
# proxmox-backup-debug recover index drive-scsi0.img.fidx /path/to/.chunks
In the above example, the `/path/to/.chunks` argument is the path to the
directory that contains the chunks, and `drive-scsi0.img.fidx` is the index file
of the file you'd like to restore. Both paths can be absolute or relative. With
``--skip-crc``, it's possible to disable the CRC checks of the chunks. This
will speed up the process slightly and allow for trying to restore (partially)
corrupt chunks. It's recommended to always try without the skip-CRC option
first.

View File

@ -41,35 +41,26 @@ Binary Data (BLOBs)
~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~
This type is used to store smaller (< 16MB) binary data such as This type is used to store smaller (< 16MB) binary data such as
configuration files. Larger files should be stored as image archives. configuration files. Larger files should be stored as image archive.
.. caution:: Please do not store all files as BLOBs. Instead, use the .. caution:: Please do not store all files as BLOBs. Instead, use the
file archive to store entire directory trees. file archive to store whole directory trees.
Catalog File: ``catalog.pcat1`` Catalog File: ``catalog.pcat1``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The catalog file is an index for file archives. It contains The catalog file is an index for file archives. It contains
the list of included files and is used to speed up search operations. the list of files and is used to speed up search operations.
The Manifest: ``index.json`` The Manifest: ``index.json``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The manifest contains a list of all backed up files, and their The manifest contains the list of all backup files, their
sizes and checksums. It is used to verify the consistency of a sizes and checksums. It is used to verify the consistency of a
backup. backup.
Backup Namespace
----------------
Namespaces allow for the reuse of a single chunk store deduplication domain for
multiple sources, while avoiding naming conflicts and getting more fine-grained
access control.
Essentially they're implemented as simple directory structure and need no
separate configuration.
Backup Type Backup Type
----------- -----------
@ -77,40 +68,38 @@ Backup Type
The backup server groups backups by *type*, where *type* is one of: The backup server groups backups by *type*, where *type* is one of:
``vm`` ``vm``
This type is used for :term:`virtual machine<Virtual machine>`\ s. It This type is used for :term:`virtual machine`\ s. Typically
typically consists of the virtual machine's configuration file and an image consists of the virtual machine's configuration file and an image archive
archive for each disk. for each disk.
``ct`` ``ct``
This type is used for :term:`container<Container>`\ s. It consists of the This type is used for :term:`container`\ s. Consists of the container's
container's configuration and a single file archive for the filesystem's configuration and a single file archive for the filesystem content.
contents.
``host`` ``host``
This type is used for file/directory backups created from within a machine. This type is used for backups created from within the backed up machine.
Typically this would be a physical host, but could also be a virtual machine Typically this would be a physical host but could also be a virtual machine
or container. Such backups may contain file and image archives; there are no or container. Such backups may contain file and image archives, there are no restrictions in this regard.
restrictions in this regard.
Backup ID Backup ID
--------- ---------
A unique ID for a specific Backup Type and Backup Namespace. Usually the A unique ID. Usually the virtual machine or container ID. ``host``
virtual machine or container ID. ``host`` type backups normally use the type backups normally use the hostname.
hostname.
Backup Time Backup Time
----------- -----------
The time when the backup was made with second resolution. The time when the backup was made.
Backup Group Backup Group
------------ ------------
The tuple ``<type>/<id>`` is called a backup group. Such a group may contain The tuple ``<type>/<ID>`` is called a backup group. Such a group
one or more backup snapshots. may contain one or more backup snapshots.
.. _term_backup_snapshot: .. _term_backup_snapshot:
@ -126,7 +115,7 @@ uniquely identifies a specific backup within a datastore.
vm/104/2019-10-09T08:01:06Z vm/104/2019-10-09T08:01:06Z
host/elsa/2019-11-08T09:48:14Z host/elsa/2019-11-08T09:48:14Z
As you can see, the time format is RFC3339_ with Coordinated As you can see, the time format is RFC3399_ with Coordinated
Universal Time (UTC_, identified by the trailing *Z*). Universal Time (UTC_, identified by the trailing *Z*).

View File

@ -1,101 +0,0 @@
.. _sysadmin_traffic_control:
Traffic Control
---------------
.. image:: images/screenshots/pbs-gui-traffic-control-add.png
:align: right
:alt: Add a traffic control limit
Creating and restoring backups can produce lots of traffic and impact other
users of the network or shared storages.
Proxmox Backup Server allows to limit network traffic for clients within
specified networks using a token bucket filter (TBF).
This allows you to avoid network congestion or to prioritize traffic from
certain hosts.
You can manage the traffic controls either over the web-interface or using the
``traffic-control`` commandos of the ``proxmox-backup-manager`` command-line
tool.
.. note:: Sync jobs on the server are not affected by its rate-in limits. If
you want to limit the incoming traffic that a pull-based sync job
generates, you need to setup a job-specific rate-in limit. See
:ref:`syncjobs`.
The following command adds a traffic control rule to limit all IPv4 clients
(network ``0.0.0.0/0``) to 100 MB/s:
.. code-block:: console
# proxmox-backup-manager traffic-control create rule0 --network 0.0.0.0/0 \
--rate-in 100MB --rate-out 100MB \
--comment "Default rate limit (100MB/s) for all clients"
.. note:: To limit both IPv4 and IPv6 network spaces you need to pass two
network parameters ``::/0`` and ``0.0.0.0/0``.
It is possible to restrict rules to certain time frames, for example the
company office hours:
.. tip:: You can use SI (base 10: KB, MB, ...) or IEC (base 2: KiB, MiB, ...)
units.
.. code-block:: console
# proxmox-backup-manager traffic-control update rule0 \
--timeframe "mon..fri 8-12" \
--timeframe "mon..fri 14:30-18"
If there are more rules, the server uses the rule with the smaller network. For
example, we can overwrite the setting for our private network (and the server
itself) with:
.. code-block:: console
# proxmox-backup-manager traffic-control create rule1 \
--network 192.168.2.0/24 \
--network 127.0.0.0/8 \
--rate-in 20GB --rate-out 20GB \
--comment "Use 20GB/s for the local network"
.. note:: The behavior is undefined if there are several rules for the same network.
If there are multiple rules that match the same network all of them will be
applied, which means that the smallest one wins, as it's bucket fills up the
fastest.
To list the current rules use:
.. code-block:: console
# proxmox-backup-manager traffic-control list
┌───────┬─────────────┬─────────────┬─────────────────────────┬────────────...─┐
│ name │ rate-in │ rate-out │ network │ timeframe ... │
╞═══════╪═════════════╪═════════════╪═════════════════════════╪════════════...═╡
│ rule0 │ 100 MB │ 100 MB │ ["0.0.0.0/0"] │ ["mon..fri ... │
├───────┼─────────────┼─────────────┼─────────────────────────┼────────────...─┤
│ rule1 │ 20 GB │ 20 GB │ ["192.168.2.0/24", ...] │ ... │
└───────┴─────────────┴─────────────┴─────────────────────────┴────────────...─┘
Rules can also be removed:
.. code-block:: console
# proxmox-backup-manager traffic-control remove rule1
To show the state (current data rate) of all configured rules use:
.. code-block:: console
# proxmox-backup-manager traffic-control traffic
┌───────┬─────────────┬──────────────┐
│ name │ cur-rate-in │ cur-rate-out │
╞═══════╪═════════════╪══════════════╡
│ rule0 │ 0 B │ 0 B │
├───────┼─────────────┼──────────────┤
│ rule1 │ 1.161 GiB │ 19.146 KiB │
└───────┴─────────────┴──────────────┘

View File

@ -15,19 +15,17 @@ Proxmox Backup Server supports several authentication realms, and you need to
choose the realm when you add a new user. Possible realms are: choose the realm when you add a new user. Possible realms are:
:pam: Linux PAM standard authentication. Use this if you want to :pam: Linux PAM standard authentication. Use this if you want to
authenticate as a Linux system user (users need to exist on the authenticate as Linux system user (Users need to exist on the
system). system).
:pbs: Proxmox Backup Server realm. This type stores hashed passwords in :pbs: Proxmox Backup Server realm. This type stores hashed passwords in
``/etc/proxmox-backup/shadow.json``. ``/etc/proxmox-backup/shadow.json``.
:openid: OpenID Connect server. Users can authenticate against an external After installation, there is a single user ``root@pam``, which
OpenID Connect server. corresponds to the Unix superuser. User configuration information is stored in the file
``/etc/proxmox-backup/user.cfg``. You can use the
After installation, there is a single user, ``root@pam``, which corresponds to ``proxmox-backup-manager`` command line tool to list or manipulate
the Unix superuser. User configuration information is stored in the file users:
``/etc/proxmox-backup/user.cfg``. You can use the ``proxmox-backup-manager``
command line tool to list or manipulate users:
.. code-block:: console .. code-block:: console
@ -42,13 +40,13 @@ command line tool to list or manipulate users:
:align: right :align: right
:alt: Add a new user :alt: Add a new user
The superuser has full administration rights on everything, so it's recommended The superuser has full administration rights on everything, so you
to add other users with less privileges. You can add a new normally want to add other users with less privileges. You can add a new
user with the ``user create`` subcommand or through the web user with the ``user create`` subcommand or through the web
interface, under the **User Management** tab of **Configuration -> Access interface, under the **User Management** tab of **Configuration -> Access
Control**. The ``create`` subcommand lets you specify many options like Control**. The ``create`` subcommand lets you specify many options like
``--email`` or ``--password``. You can update or change any user properties ``--email`` or ``--password``. You can update or change any user properties
using the ``user update`` subcommand later (**Edit** in the GUI): using the ``update`` subcommand later (**Edit** in the GUI):
.. code-block:: console .. code-block:: console
@ -73,16 +71,16 @@ The resulting user list looks like this:
│ root@pam │ 1 │ │ │ │ │ Superuser │ │ root@pam │ 1 │ │ │ │ │ Superuser │
└──────────┴────────┴────────┴───────────┴──────────┴──────────────────┴──────────────────┘ └──────────┴────────┴────────┴───────────┴──────────┴──────────────────┴──────────────────┘
Newly created users do not have any permissions. Please read the :ref:`user_acl` Newly created users do not have any permissions. Please read the Access Control
section to learn how to set access permissions. section to learn how to set access permissions.
You can disable a user account by setting ``--enable`` to ``0``: If you want to disable a user account, you can do that by setting ``--enable`` to ``0``
.. code-block:: console .. code-block:: console
# proxmox-backup-manager user update john@pbs --enable 0 # proxmox-backup-manager user update john@pbs --enable 0
Or completely remove a user with: Or completely remove the user with:
.. code-block:: console .. code-block:: console
@ -97,7 +95,7 @@ API Tokens
:align: right :align: right
:alt: API Token Overview :alt: API Token Overview
Any authenticated user can generate API tokens, which can in turn be used to Any authenticated user can generate API tokens which can in turn be used to
configure various clients, instead of directly providing the username and configure various clients, instead of directly providing the username and
password. password.
@ -119,7 +117,7 @@ The API token is passed from the client to the server by setting the
``Authorization`` HTTP header with method ``PBSAPIToken`` to the value ``Authorization`` HTTP header with method ``PBSAPIToken`` to the value
``TOKENID:TOKENSECRET``. ``TOKENID:TOKENSECRET``.
You can generate tokens from the GUI or by using ``proxmox-backup-manager``: Generating new tokens can done using ``proxmox-backup-manager`` or the GUI:
.. code-block:: console .. code-block:: console
@ -156,134 +154,35 @@ section to learn how to set access permissions.
Access Control Access Control
-------------- --------------
By default, new users and API tokens do not have any permissions. Instead you By default new users and API tokens do not have any permission. Instead you
need to specify what is allowed and what is not. need to specify what is allowed and what is not. You can do this by assigning
roles to users/tokens on specific objects like datastores or remotes. The
Proxmox Backup Server uses a role and path based permission management system. following roles exist:
An entry in the permissions table allows a user, group or token to take on a
specific role when accessing an 'object' or 'path'. This means that such an
access rule can be represented as a triple of '(path, user, role)', '(path,
group, role)' or '(path, token, role)', with the role containing a set of
allowed actions, and the path representing the target of these actions.
Privileges
~~~~~~~~~~
Privileges are the atoms that access roles are made off. They are internally
used to enforce the actual permission checks in the API.
We currently support the following privileges:
**Sys.Audit**
Sys.Audit allows one to know about the system and its status.
**Sys.Modify**
Sys.Modify allows one to modify system-level configuration and apply updates.
**Sys.PowerManagement**
Sys.Modify allows one to to poweroff or reboot the system.
**Datastore.Audit**
Datastore.Audit allows one to know about a datastore, including reading the
configuration entry and listing its contents.
**Datastore.Allocate**
Datastore.Allocate allows one to create or deleting datastores.
**Datastore.Modify**
Datastore.Modify allows one to modify a datastore and its contents, and to
create or delete namespaces inside a datastore.
**Datastore.Read**
Datastore.Read allows one to read arbitrary backup contents, independent of
the backup group owner.
**Datastore.Verify**
Allows verifying the backup snapshots in a datastore.
**Datastore.Backup**
Datastore.Backup allows one create new backup snapshot and gives one also the
privileges of Datastore.Read and Datastore.Verify, but only if the backup
group is owned by the user or one of its tokens.
**Datastore.Prune**
Datastore.Prune allows one to delete snapshots, but additionally requires
backup ownership
**Permissions.Modify**
Permissions.Modify allows one to modifying ACLs
.. note:: One can always configure privileges for their own API tokens, as
they will clamped by the users privileges anyway.
**Remote.Audit**
Remote.Audit allows one to read the remote and the sync configuration entries
**Remote.Modify**
Remote.Modify allows one to modify the remote configuration
**Remote.Read**
Remote.Read allows one to read data from a configured `Remote`
**Sys.Console**
Sys.Console allows one to access to the system's console, note that for all
but `root@pam` a valid system login is still required.
**Tape.Audit**
Tape.Audit allows one to read the configuration and status of tape drives,
changers and backups
**Tape.Modify**
Tape.Modify allows one to modify the configuration of tape drives, changers
and backups
**Tape.Write**
Tape.Write allows one to write to a tape media
**Tape.Read**
Tape.Read allows one to read tape backup configuration and contents from a
tape media
**Realm.Allocate**
Realm.Allocate allows one to view, create, modify and delete authentication
realms for users
Access Roles
~~~~~~~~~~~~
An access role combines one or more privileges into something that can be
assigned to an user or API token on an object path.
Currently there are only built-in roles, that means, you cannot create your
own, custom role.
The following roles exist:
**NoAccess** **NoAccess**
Disable Access - nothing is allowed. Disable Access - nothing is allowed.
**Admin** **Admin**
Can do anything, on the object path assigned. Can do anything.
**Audit** **Audit**
Can view the status and configuration of things, but is not allowed to change Can view things, but is not allowed to change settings.
settings.
**DatastoreAdmin** **DatastoreAdmin**
Can do anything on *existing* datastores. Can do anything on datastores.
**DatastoreAudit** **DatastoreAudit**
Can view datastore metrics, settings and list content. But is not allowed to Can view datastore settings and list content. But
read the actual data. is not allowed to read the actual data.
**DatastoreReader** **DatastoreReader**
Can inspect a datastore's or namespaces content and do restores. Can Inspect datastore content and can do restores.
**DatastoreBackup** **DatastoreBackup**
Can backup and restore owned backups. Can backup and restore owned backups.
**DatastorePowerUser** **DatastorePowerUser**
Can backup, restore, and prune *owned* backups. Can backup, restore, and prune owned backups.
**RemoteAdmin** **RemoteAdmin**
Can do anything on remotes. Can do anything on remotes.
@ -294,62 +193,7 @@ The following roles exist:
**RemoteSyncOperator** **RemoteSyncOperator**
Is allowed to read data from a remote. Is allowed to read data from a remote.
**TapeAdmin** .. image:: images/screenshots/pbs-gui-user-management-add-user.png
Can do anything related to tape backup
**TapeAudit**
Can view tape related metrics, configuration and status
**TapeOperator**
Can do tape backup and restore, but cannot change any configuration
**TapeReader**
Can read and inspect tape configuration and media content
Objects and Paths
~~~~~~~~~~~~~~~~~
Access permissions are assigned to objects, such as a datastore, a namespace or
some system resources.
We use file system like paths to address these objects. These paths form a
natural tree, and permissions of higher levels (shorter paths) can optionally
be propagated down within this hierarchy.
Paths can be templated, that means they can refer to the actual id of an
configuration entry. When an API call requires permissions on a templated
path, the path may contain references to parameters of the API call. These
references are specified in curly braces.
Some examples are:
* `/datastore`: Access to *all* datastores on a Proxmox Backup server
* `/datastore/{store}`: Access to a specific datastore on a Proxmox Backup
server
* `/datastore/{store}/{ns}`: Access to a specific namespace on a specific
datastore
* `/remote`: Access to all remote entries
* `/system/network`: Access to configuring the host network
* `/tape/`: Access to tape devices, pools and jobs
* `/access/users`: User administration
* `/access/openid/{id}`: Administrative access to a specific OpenID Connect realm
Inheritance
^^^^^^^^^^^
As mentioned earlier, object paths form a file system like tree, and
permissions can be inherited by objects down that tree through the propagate
flag, which is set by default. We use the following inheritance rules:
* Permissions for API tokens are always clamped to the one of the user.
* Permissions on deeper, more specific levels replace those inherited from an
upper level.
Configuration & Management
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. image:: images/screenshots/pbs-gui-permissions-add.png
:align: right :align: right
:alt: Add permissions for user :alt: Add permissions for user
@ -392,8 +236,7 @@ You can list the ACLs of each user/token using the following command:
│ john@pbs │ /datastore/store1 │ 1 │ DatastoreAdmin │ │ john@pbs │ /datastore/store1 │ 1 │ DatastoreAdmin │
└──────────┴───────────────────┴───────────┴────────────────┘ └──────────┴───────────────────┴───────────┴────────────────┘
A single user/token can be assigned multiple permission sets for different A single user/token can be assigned multiple permission sets for different datastores.
datastores.
.. Note:: .. Note::
Naming convention is important here. For datastores on the host, Naming convention is important here. For datastores on the host,
@ -404,11 +247,11 @@ datastores.
remote (see `Remote` below) and ``{storename}`` is the name of the datastore on remote (see `Remote` below) and ``{storename}`` is the name of the datastore on
the remote. the remote.
API Token Permissions API Token permissions
~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~
API token permissions are calculated based on ACLs containing their ID, API token permissions are calculated based on ACLs containing their ID
independently of those of their corresponding user. The resulting permission set independent of those of their corresponding user. The resulting permission set
on a given path is then intersected with that of the corresponding user. on a given path is then intersected with that of the corresponding user.
In practice this means: In practice this means:
@ -416,10 +259,10 @@ In practice this means:
#. API tokens require their own ACL entries #. API tokens require their own ACL entries
#. API tokens can never do more than their corresponding user #. API tokens can never do more than their corresponding user
Effective Permissions Effective permissions
~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~
To calculate and display the effective permission set of a user or API token, To calculate and display the effective permission set of a user or API token
you can use the ``proxmox-backup-manager user permission`` command: you can use the ``proxmox-backup-manager user permission`` command:
.. code-block:: console .. code-block:: console
@ -444,7 +287,7 @@ you can use the ``proxmox-backup-manager user permission`` command:
.. _user_tfa: .. _user_tfa:
Two-Factor Authentication Two-factor authentication
------------------------- -------------------------
Introduction Introduction
@ -453,7 +296,7 @@ Introduction
With simple authentication, only a password (single factor) is required to With simple authentication, only a password (single factor) is required to
successfully claim an identity (authenticate), for example, to be able to log in successfully claim an identity (authenticate), for example, to be able to log in
as `root@pam` on a specific instance of Proxmox Backup Server. In this case, if as `root@pam` on a specific instance of Proxmox Backup Server. In this case, if
the password gets leaked or stolen, anybody can use it to log in - even if they the password gets stolen or leaked, anybody can use it to log in - even if they
should not be allowed to do so. should not be allowed to do so.
With two-factor authentication (TFA), a user is asked for an additional factor With two-factor authentication (TFA), a user is asked for an additional factor
@ -516,18 +359,16 @@ WebAuthn
For WebAuthn to work, you need to have two things: For WebAuthn to work, you need to have two things:
* A trusted HTTPS certificate (for example, by using `Let's Encrypt * a trusted HTTPS certificate (for example, by using `Let's Encrypt
<https://pbs.proxmox.com/wiki/index.php/HTTPS_Certificate_Configuration>`_). <https://pbs.proxmox.com/wiki/index.php/HTTPS_Certificate_Configuration>`_).
While it probably works with an untrusted certificate, some browsers may warn While it probably works with an untrusted certificate, some browsers may warn
or refuse WebAuthn operations if it is not trusted. or refuse WebAuthn operations if it is not trusted.
* Setup the WebAuthn configuration (see **Configuration -> Authentication** in * setup the WebAuthn configuration (see *Configuration -> Authentication* in the
the Proxmox Backup Server web interface). This can be auto-filled in most Proxmox Backup Server web-interface). This can be auto-filled in most setups.
setups.
Once you have fulfilled both of these requirements, you can add a WebAuthn Once you have fulfilled both of these requirements, you can add a WebAuthn
configuration in the **Two Factor Authentication** tab of the **Access Control** configuration in the *Access Control* panel.
panel.
.. _user_tfa_setup_recovery_keys: .. _user_tfa_setup_recovery_keys:
@ -539,8 +380,7 @@ Recovery Keys
:alt: Add a new user :alt: Add a new user
Recovery key codes do not need any preparation; you can simply create a set of Recovery key codes do not need any preparation; you can simply create a set of
recovery keys in the **Two Factor Authentication** tab of the **Access Control** recovery keys in the *Access Control* panel.
panel.
.. note:: There can only be one set of single-use recovery keys per user at any .. note:: There can only be one set of single-use recovery keys per user at any
time. time.

View File

@ -1 +1 @@
deb https://enterprise.proxmox.com/debian/pbs bullseye pbs-enterprise deb https://enterprise.proxmox.com/debian/pbs buster pbs-enterprise

View File

@ -1,8 +1,9 @@
use anyhow::Error; use anyhow::{Error};
// chacha20-poly1305 // chacha20-poly1305
fn rate_test(name: &str, bench: &dyn Fn() -> usize) { fn rate_test(name: &str, bench: &dyn Fn() -> usize) {
print!("{:<20} ", name); print!("{:<20} ", name);
let start = std::time::SystemTime::now(); let start = std::time::SystemTime::now();
@ -13,19 +14,20 @@ fn rate_test(name: &str, bench: &dyn Fn() -> usize) {
loop { loop {
bytes += bench(); bytes += bench();
let elapsed = start.elapsed().unwrap(); let elapsed = start.elapsed().unwrap();
if elapsed > duration { if elapsed > duration { break; }
break;
}
} }
let elapsed = start.elapsed().unwrap(); let elapsed = start.elapsed().unwrap();
let elapsed = (elapsed.as_secs() as f64) + (elapsed.subsec_millis() as f64) / 1000.0; let elapsed = (elapsed.as_secs() as f64) +
(elapsed.subsec_millis() as f64)/1000.0;
println!("{:>8.1} MB/s", (bytes as f64) / (elapsed * 1024.0 * 1024.0)); println!("{:>8.1} MB/s", (bytes as f64)/(elapsed*1024.0*1024.0));
} }
fn main() -> Result<(), Error> { fn main() -> Result<(), Error> {
let input = proxmox_sys::linux::random_data(1024 * 1024)?;
let input = proxmox::sys::linux::random_data(1024*1024)?;
rate_test("crc32", &|| { rate_test("crc32", &|| {
let mut crchasher = crc32fast::Hasher::new(); let mut crchasher = crc32fast::Hasher::new();
@ -44,23 +46,35 @@ fn main() -> Result<(), Error> {
input.len() input.len()
}); });
let key = proxmox_sys::linux::random_data(32)?; let key = proxmox::sys::linux::random_data(32)?;
let iv = proxmox_sys::linux::random_data(16)?; let iv = proxmox::sys::linux::random_data(16)?;
let cipher = openssl::symm::Cipher::aes_256_gcm(); let cipher = openssl::symm::Cipher::aes_256_gcm();
rate_test("aes-256-gcm", &|| { rate_test("aes-256-gcm", &|| {
let mut tag = [0u8; 16]; let mut tag = [0u8;16];
openssl::symm::encrypt_aead(cipher, &key, Some(&iv), b"", &input, &mut tag).unwrap(); openssl::symm::encrypt_aead(
cipher,
&key,
Some(&iv),
b"",
&input,
&mut tag).unwrap();
input.len() input.len()
}); });
let cipher = openssl::symm::Cipher::chacha20_poly1305(); let cipher = openssl::symm::Cipher::chacha20_poly1305();
rate_test("chacha20-poly1305", &|| { rate_test("chacha20-poly1305", &|| {
let mut tag = [0u8; 16]; let mut tag = [0u8;16];
openssl::symm::encrypt_aead(cipher, &key, Some(&iv[..12]), b"", &input, &mut tag).unwrap(); openssl::symm::encrypt_aead(
cipher,
&key,
Some(&iv[..12]),
b"",
&input,
&mut tag).unwrap();
input.len() input.len()
}); });

View File

@ -1,7 +1,6 @@
use anyhow::Error; use anyhow::{Error};
use proxmox_router::cli::*; use proxmox::api::{*, cli::*};
use proxmox_schema::*;
#[api( #[api(
input: { input: {
@ -16,7 +15,9 @@ use proxmox_schema::*;
/// Echo command. Print the passed text. /// Echo command. Print the passed text.
/// ///
/// Returns: nothing /// Returns: nothing
fn echo_command(text: String) -> Result<(), Error> { fn echo_command(
text: String,
) -> Result<(), Error> {
println!("{}", text); println!("{}", text);
Ok(()) Ok(())
} }
@ -35,7 +36,9 @@ fn echo_command(text: String) -> Result<(), Error> {
/// Hello command. /// Hello command.
/// ///
/// Returns: nothing /// Returns: nothing
fn hello_command(verbose: Option<bool>) -> Result<(), Error> { fn hello_command(
verbose: Option<bool>,
) -> Result<(), Error> {
if verbose.unwrap_or(false) { if verbose.unwrap_or(false) {
println!("Hello, how are you!"); println!("Hello, how are you!");
} else { } else {
@ -50,6 +53,7 @@ fn hello_command(verbose: Option<bool>) -> Result<(), Error> {
/// ///
/// Returns: nothing /// Returns: nothing
fn quit_command() -> Result<(), Error> { fn quit_command() -> Result<(), Error> {
println!("Goodbye."); println!("Goodbye.");
std::process::exit(0); std::process::exit(0);
@ -59,9 +63,8 @@ fn cli_definition() -> CommandLineInterface {
let cmd_def = CliCommandMap::new() let cmd_def = CliCommandMap::new()
.insert("quit", CliCommand::new(&API_METHOD_QUIT_COMMAND)) .insert("quit", CliCommand::new(&API_METHOD_QUIT_COMMAND))
.insert("hello", CliCommand::new(&API_METHOD_HELLO_COMMAND)) .insert("hello", CliCommand::new(&API_METHOD_HELLO_COMMAND))
.insert( .insert("echo", CliCommand::new(&API_METHOD_ECHO_COMMAND)
"echo", .arg_param(&["text"])
CliCommand::new(&API_METHOD_ECHO_COMMAND).arg_param(&["text"]),
) )
.insert_help(); .insert_help();
@ -69,6 +72,7 @@ fn cli_definition() -> CommandLineInterface {
} }
fn main() -> Result<(), Error> { fn main() -> Result<(), Error> {
let helper = CliHelper::new(cli_definition()); let helper = CliHelper::new(cli_definition());
let mut rl = rustyline::Editor::<CliHelper>::new(); let mut rl = rustyline::Editor::<CliHelper>::new();

View File

@ -1,15 +1,16 @@
use std::io::Write; use std::io::Write;
use anyhow::Error; use anyhow::{Error};
use pbs_api_types::{Authid, BackupNamespace, BackupType}; use proxmox_backup::api2::types::Authid;
use pbs_client::{BackupReader, HttpClient, HttpClientOptions}; use proxmox_backup::client::{HttpClient, HttpClientOptions, BackupReader};
pub struct DummyWriter { pub struct DummyWriter {
bytes: usize, bytes: usize,
} }
impl Write for DummyWriter { impl Write for DummyWriter {
fn write(&mut self, data: &[u8]) -> Result<usize, std::io::Error> { fn write(&mut self, data: &[u8]) -> Result<usize, std::io::Error> {
self.bytes += data.len(); self.bytes += data.len();
Ok(data.len()) Ok(data.len())
@ -20,7 +21,9 @@ impl Write for DummyWriter {
} }
} }
async fn run() -> Result<(), Error> { async fn run() -> Result<(), Error> {
let host = "localhost"; let host = "localhost";
let auth_id = Authid::root_auth_id(); let auth_id = Authid::root_auth_id();
@ -31,17 +34,10 @@ async fn run() -> Result<(), Error> {
let client = HttpClient::new(host, 8007, auth_id, options)?; let client = HttpClient::new(host, 8007, auth_id, options)?;
let backup_time = proxmox_time::parse_rfc3339("2019-06-28T10:49:48Z")?; let backup_time = proxmox::tools::time::parse_rfc3339("2019-06-28T10:49:48Z")?;
let client = BackupReader::start( let client = BackupReader::start(client, None, "store2", "host", "elsa", backup_time, true)
client, .await?;
None,
"store2",
&BackupNamespace::root(),
&(BackupType::Host, "elsa".to_string(), backup_time).into(),
true,
)
.await?;
let start = std::time::SystemTime::now(); let start = std::time::SystemTime::now();
@ -54,19 +50,16 @@ async fn run() -> Result<(), Error> {
} }
let elapsed = start.elapsed().unwrap(); let elapsed = start.elapsed().unwrap();
let elapsed = (elapsed.as_secs() as f64) + (elapsed.subsec_millis() as f64) / 1000.0; let elapsed = (elapsed.as_secs() as f64) +
(elapsed.subsec_millis() as f64)/1000.0;
println!( println!("Downloaded {} bytes, {} MB/s", bytes, (bytes as f64)/(elapsed*1024.0*1024.0));
"Downloaded {} bytes, {} MB/s",
bytes,
(bytes as f64) / (elapsed * 1024.0 * 1024.0)
);
Ok(()) Ok(())
} }
fn main() { fn main() {
if let Err(err) = proxmox_async::runtime::main(run()) { if let Err(err) = proxmox_backup::tools::runtime::main(run()) {
eprintln!("ERROR: {}", err); eprintln!("ERROR: {}", err);
} }
println!("DONE"); println!("DONE");

View File

@ -1,9 +1,9 @@
use std::io::Write;
use std::path::PathBuf;
use std::thread;
use anyhow::{bail, Error}; use anyhow::{bail, Error};
use std::thread;
use std::path::PathBuf;
use std::io::Write;
// tar handle files that shrink during backup, by simply padding with zeros. // tar handle files that shrink during backup, by simply padding with zeros.
// //
// this binary run multiple thread which writes some large files, then truncates // this binary run multiple thread which writes some large files, then truncates
@ -19,15 +19,15 @@ use anyhow::{bail, Error};
// Error: detected shrunk file "./dyntest1/testfile0.dat" (22020096 < 12679380992) // Error: detected shrunk file "./dyntest1/testfile0.dat" (22020096 < 12679380992)
fn create_large_file(path: PathBuf) { fn create_large_file(path: PathBuf) {
println!("TEST {:?}", path); println!("TEST {:?}", path);
let mut file = std::fs::OpenOptions::new() let mut file = std::fs::OpenOptions::new()
.write(true) .write(true)
.create_new(true) .create_new(true)
.open(&path) .open(&path).unwrap();
.unwrap();
let buffer = vec![0u8; 64 * 1024]; let buffer = vec![0u8; 64*1024];
loop { loop {
for _ in 0..64 { for _ in 0..64 {
@ -40,6 +40,7 @@ fn create_large_file(path: PathBuf) {
} }
fn main() -> Result<(), Error> { fn main() -> Result<(), Error> {
let base = PathBuf::from("dyntest1"); let base = PathBuf::from("dyntest1");
let _ = std::fs::create_dir(&base); let _ = std::fs::create_dir(&base);

View File

@ -69,7 +69,7 @@ fn send_request(
} }
fn main() -> Result<(), Error> { fn main() -> Result<(), Error> {
proxmox_async::runtime::main(run()) proxmox_backup::tools::runtime::main(run())
} }
async fn run() -> Result<(), Error> { async fn run() -> Result<(), Error> {

Some files were not shown because too many files have changed in this diff Show More