1459 lines
61 KiB
ReStructuredText
1459 lines
61 KiB
ReStructuredText
Backup Management
|
|
=================
|
|
|
|
.. The administration guide.
|
|
.. todo:: either add a bit more explanation or remove the previous sentence
|
|
|
|
Terminology
|
|
-----------
|
|
|
|
Backup Content
|
|
~~~~~~~~~~~~~~
|
|
|
|
When doing deduplication, there are different strategies to get
|
|
optimal results in terms of performance and/or deduplication rates.
|
|
Depending on the type of data, it can be split into *fixed* or *variable*
|
|
sized chunks.
|
|
|
|
Fixed sized chunking requires minimal CPU power, and is used to
|
|
backup virtual machine images.
|
|
|
|
Variable sized chunking needs more CPU power, but is essential to get
|
|
good deduplication rates for file archives.
|
|
|
|
The Proxmox Backup Server supports both strategies.
|
|
|
|
|
|
Image Archives: ``<name>.img``
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
This is used for virtual machine images and other large binary
|
|
data. Content is split into fixed-sized chunks.
|
|
|
|
|
|
File Archives: ``<name>.pxar``
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
.. see https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/
|
|
|
|
A file archive stores a full directory tree. Content is stored using
|
|
the :ref:`pxar-format`, split into variable-sized chunks. The format
|
|
is optimized to achieve good deduplication rates.
|
|
|
|
|
|
Binary Data (BLOBs)
|
|
^^^^^^^^^^^^^^^^^^^
|
|
|
|
This type is used to store smaller (< 16MB) binary data such as
|
|
configuration files. Larger files should be stored as image archive.
|
|
|
|
.. caution:: Please do not store all files as BLOBs. Instead, use the
|
|
file archive to store whole directory trees.
|
|
|
|
|
|
Catalog File: ``catalog.pcat1``
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
The catalog file is an index for file archives. It contains
|
|
the list of files and is used to speed up search operations.
|
|
|
|
|
|
The Manifest: ``index.json``
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
The manifest contains the list of all backup files, their
|
|
sizes and checksums. It is used to verify the consistency of a
|
|
backup.
|
|
|
|
|
|
Backup Type
|
|
~~~~~~~~~~~
|
|
|
|
The backup server groups backups by *type*, where *type* is one of:
|
|
|
|
``vm``
|
|
This type is used for :term:`virtual machine`\ s. Typically
|
|
consists of the virtual machine's configuration file and an image archive
|
|
for each disk.
|
|
|
|
``ct``
|
|
This type is used for :term:`container`\ s. Consists of the container's
|
|
configuration and a single file archive for the filesystem content.
|
|
|
|
``host``
|
|
This type is used for backups created from within the backed up machine.
|
|
Typically this would be a physical host but could also be a virtual machine
|
|
or container. Such backups may contain file and image archives, there are no restrictions in this regard.
|
|
|
|
|
|
Backup ID
|
|
~~~~~~~~~
|
|
|
|
A unique ID. Usually the virtual machine or container ID. ``host``
|
|
type backups normally use the hostname.
|
|
|
|
|
|
Backup Time
|
|
~~~~~~~~~~~
|
|
|
|
The time when the backup was made.
|
|
|
|
|
|
Backup Group
|
|
~~~~~~~~~~~~
|
|
|
|
The tuple ``<type>/<ID>`` is called a backup group. Such a group
|
|
may contain one or more backup snapshots.
|
|
|
|
|
|
Backup Snapshot
|
|
~~~~~~~~~~~~~~~
|
|
|
|
The triplet ``<type>/<ID>/<time>`` is called a backup snapshot. It
|
|
uniquely identifies a specific backup within a datastore.
|
|
|
|
.. code-block:: console
|
|
:caption: Backup Snapshot Examples
|
|
|
|
vm/104/2019-10-09T08:01:06Z
|
|
host/elsa/2019-11-08T09:48:14Z
|
|
|
|
As you can see, the time format is RFC3399_ with Coordinated
|
|
Universal Time (UTC_, identified by the trailing *Z*).
|
|
|
|
Backup Server Management
|
|
------------------------
|
|
|
|
The command line tool to configure and manage the backup server is called
|
|
:command:`proxmox-backup-manager`.
|
|
|
|
|
|
|
|
:term:`DataStore`
|
|
~~~~~~~~~~~~~~~~~
|
|
|
|
A datastore refers to a location at which backups are stored. The current
|
|
implementation uses a directory inside a standard unix file system (``ext4``,
|
|
``xfs`` or ``zfs``) to store the backup data.
|
|
|
|
Datastores are identified by a simple *ID*. You can configure this
|
|
when setting up the datastore. The configuration information for datastores
|
|
is stored in the file ``/etc/proxmox-backup/datastore.cfg``.
|
|
|
|
.. note:: The `File Layout`_ requires the file system to support at least *65538*
|
|
subdirectories per directory. That number comes from the 2\ :sup:`16`
|
|
pre-created chunk namespace directories, and the ``.`` and ``..`` default
|
|
directory entries. This requirement excludes certain filesystems and
|
|
filesystem configuration from being supported for a datastore. For example,
|
|
``ext3`` as a whole or ``ext4`` with the ``dir_nlink`` feature manually disabled.
|
|
|
|
Disk Management
|
|
~~~~~~~~~~~~~~~
|
|
|
|
.. image:: images/screenshots/pbs-gui-disks.png
|
|
:align: right
|
|
:alt: List of disks
|
|
|
|
Proxmox Backup Server comes with a set of disk utilities, which are
|
|
accessed using the ``disk`` subcommand. This subcommand allows you to initialize
|
|
disks, create various filesystems, and get information about the disks.
|
|
|
|
To view the disks connected to the system, navigate to **Administration ->
|
|
Disks** in the web interface or use the ``list`` subcommand of
|
|
``disk``:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager disk list
|
|
┌──────┬────────┬─────┬───────────┬─────────────┬───────────────┬─────────┬────────┐
|
|
│ name │ used │ gpt │ disk-type │ size │ model │ wearout │ status │
|
|
╞══════╪════════╪═════╪═══════════╪═════════════╪═══════════════╪═════════╪════════╡
|
|
│ sda │ lvm │ 1 │ hdd │ 34359738368 │ QEMU_HARDDISK │ - │ passed │
|
|
├──────┼────────┼─────┼───────────┼─────────────┼───────────────┼─────────┼────────┤
|
|
│ sdb │ unused │ 1 │ hdd │ 68719476736 │ QEMU_HARDDISK │ - │ passed │
|
|
├──────┼────────┼─────┼───────────┼─────────────┼───────────────┼─────────┼────────┤
|
|
│ sdc │ unused │ 1 │ hdd │ 68719476736 │ QEMU_HARDDISK │ - │ passed │
|
|
└──────┴────────┴─────┴───────────┴─────────────┴───────────────┴─────────┴────────┘
|
|
|
|
To initialize a disk with a new GPT, use the ``initialize`` subcommand:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager disk initialize sdX
|
|
|
|
.. image:: images/screenshots/pbs-gui-disks-dir-create.png
|
|
:align: right
|
|
:alt: Create a directory
|
|
|
|
You can create an ``ext4`` or ``xfs`` filesystem on a disk using ``fs
|
|
create``, or by navigating to **Administration -> Disks -> Directory** in the
|
|
web interface and creating one from there. The following command creates an
|
|
``ext4`` filesystem and passes the ``--add-datastore`` parameter, in order to
|
|
automatically create a datastore on the disk (in this case ``sdd``). This will
|
|
create a datastore at the location ``/mnt/datastore/store1``:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager disk fs create store1 --disk sdd --filesystem ext4 --add-datastore true
|
|
|
|
.. image:: images/screenshots/pbs-gui-disks-zfs-create.png
|
|
:align: right
|
|
:alt: Create ZFS
|
|
|
|
You can also create a ``zpool`` with various raid levels from **Administration
|
|
-> Disks -> Zpool** in the web interface, or by using ``zpool create``. The command
|
|
below creates a mirrored ``zpool`` using two disks (``sdb`` & ``sdc``) and
|
|
mounts it on the root directory (default):
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager disk zpool create zpool1 --devices sdb,sdc --raidlevel mirror
|
|
|
|
.. note:: You can also pass the ``--add-datastore`` parameter here, to automatically
|
|
create a datastore from the disk.
|
|
|
|
You can use ``disk fs list`` and ``disk zpool list`` to keep track of your
|
|
filesystems and zpools respectively.
|
|
|
|
Proxmox Backup Server uses the package smartmontools. This is a set of tools
|
|
used to monitor and control the S.M.A.R.T. system for local hard disks. If a
|
|
disk supports S.M.A.R.T. capability, and you have this enabled, you can
|
|
display S.M.A.R.T. attributes from the web interface or by using the command:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager disk smart-attributes sdX
|
|
|
|
.. note:: This functionality may also be accessed directly through the use of
|
|
the ``smartctl`` command, which comes as part of the smartmontools package
|
|
(see ``man smartctl`` for more details).
|
|
|
|
|
|
Datastore Configuration
|
|
~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. image:: images/screenshots/pbs-gui-datastore.png
|
|
:align: right
|
|
:alt: Datastore Overview
|
|
|
|
You can configure multiple datastores. Minimum one datastore needs to be
|
|
configured. The datastore is identified by a simple *name* and points to a
|
|
directory on the filesystem. Each datastore also has associated retention
|
|
settings of how many backup snapshots for each interval of ``hourly``,
|
|
``daily``, ``weekly``, ``monthly``, ``yearly`` as well as a time-independent
|
|
number of backups to keep in that store. :ref:`Pruning <pruning>` and
|
|
:ref:`garbage collection <garbage-collection>` can also be configured to run
|
|
periodically based on a configured schedule (see :ref:`calendar-events`) per datastore.
|
|
|
|
Creating a Datastore
|
|
^^^^^^^^^^^^^^^^^^^^
|
|
.. image:: images/screenshots/pbs-gui-datastore-create-general.png
|
|
:align: right
|
|
:alt: Create a datastore
|
|
|
|
You can create a new datastore from the web GUI, by navigating to **Datastore** in
|
|
the menu tree and clicking **Create**. Here:
|
|
|
|
* *Name* refers to the name of the datastore
|
|
* *Backing Path* is the path to the directory upon which you want to create the
|
|
datastore
|
|
* *GC Schedule* refers to the time and intervals at which garbage collection
|
|
runs
|
|
* *Prune Schedule* refers to the frequency at which pruning takes place
|
|
* *Prune Options* set the amount of backups which you would like to keep (see :ref:`Pruning <pruning>`).
|
|
|
|
Alternatively you can create a new datastore from the command line. The
|
|
following command creates a new datastore called ``store1`` on :file:`/backup/disk1/store1`
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager datastore create store1 /backup/disk1/store1
|
|
|
|
Managing Datastores
|
|
^^^^^^^^^^^^^^^^^^^
|
|
|
|
To list existing datastores from the command line run:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager datastore list
|
|
┌────────┬──────────────────────┬─────────────────────────────┐
|
|
│ name │ path │ comment │
|
|
╞════════╪══════════════════════╪═════════════════════════════╡
|
|
│ store1 │ /backup/disk1/store1 │ This is my default storage. │
|
|
└────────┴──────────────────────┴─────────────────────────────┘
|
|
|
|
You can change the garbage collection and prune settings of a datastore, by
|
|
editing the datastore from the GUI or by using the ``update`` subcommand. For
|
|
example, the below command changes the garbage collection schedule using the
|
|
``update`` subcommand and prints the properties of the datastore with the
|
|
``show`` subcommand:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager datastore update store1 --gc-schedule 'Tue 04:27'
|
|
# proxmox-backup-manager datastore show store1
|
|
┌────────────────┬─────────────────────────────┐
|
|
│ Name │ Value │
|
|
╞════════════════╪═════════════════════════════╡
|
|
│ name │ store1 │
|
|
├────────────────┼─────────────────────────────┤
|
|
│ path │ /backup/disk1/store1 │
|
|
├────────────────┼─────────────────────────────┤
|
|
│ comment │ This is my default storage. │
|
|
├────────────────┼─────────────────────────────┤
|
|
│ gc-schedule │ Tue 04:27 │
|
|
├────────────────┼─────────────────────────────┤
|
|
│ keep-last │ 7 │
|
|
├────────────────┼─────────────────────────────┤
|
|
│ prune-schedule │ daily │
|
|
└────────────────┴─────────────────────────────┘
|
|
|
|
Finally, it is possible to remove the datastore configuration:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager datastore remove store1
|
|
|
|
.. note:: The above command removes only the datastore configuration. It does
|
|
not delete any data from the underlying directory.
|
|
|
|
|
|
File Layout
|
|
^^^^^^^^^^^
|
|
|
|
After creating a datastore, the following default layout will appear:
|
|
|
|
.. code-block:: console
|
|
|
|
# ls -arilh /backup/disk1/store1
|
|
276493 -rw-r--r-- 1 backup backup 0 Jul 8 12:35 .lock
|
|
276490 drwxr-x--- 1 backup backup 1064960 Jul 8 12:35 .chunks
|
|
|
|
`.lock` is an empty file used for process locking.
|
|
|
|
The `.chunks` directory contains folders, starting from `0000` and taking hexadecimal values until `ffff`. These
|
|
directories will store the chunked data after a backup operation has been executed.
|
|
|
|
.. code-block:: console
|
|
|
|
# ls -arilh /backup/disk1/store1/.chunks
|
|
545824 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 ffff
|
|
545823 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffe
|
|
415621 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffd
|
|
415620 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffc
|
|
353187 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffb
|
|
344995 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffa
|
|
144079 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff9
|
|
144078 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff8
|
|
144077 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff7
|
|
...
|
|
403180 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000c
|
|
403179 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000b
|
|
403177 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000a
|
|
402530 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0009
|
|
402513 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0008
|
|
402509 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0007
|
|
276509 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0006
|
|
276508 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0005
|
|
276507 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0004
|
|
276501 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0003
|
|
276499 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0002
|
|
276498 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0001
|
|
276494 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0000
|
|
276489 drwxr-xr-x 3 backup backup 4.0K Jul 8 12:35 ..
|
|
276490 drwxr-x--- 1 backup backup 1.1M Jul 8 12:35 .
|
|
|
|
|
|
|
|
User Management
|
|
~~~~~~~~~~~~~~~
|
|
|
|
.. image:: images/screenshots/pbs-gui-user-management.png
|
|
:align: right
|
|
:alt: User management
|
|
|
|
Proxmox Backup Server supports several authentication realms, and you need to
|
|
choose the realm when you add a new user. Possible realms are:
|
|
|
|
:pam: Linux PAM standard authentication. Use this if you want to
|
|
authenticate as Linux system user (Users need to exist on the
|
|
system).
|
|
|
|
:pbs: Proxmox Backup Server realm. This type stores hashed passwords in
|
|
``/etc/proxmox-backup/shadow.json``.
|
|
|
|
After installation, there is a single user ``root@pam``, which
|
|
corresponds to the Unix superuser. User configuration information is stored in the file
|
|
``/etc/proxmox-backup/user.cfg``. You can use the
|
|
``proxmox-backup-manager`` command line tool to list or manipulate
|
|
users:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager user list
|
|
┌─────────────┬────────┬────────┬───────────┬──────────┬────────────────┬────────────────────┐
|
|
│ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
|
|
╞═════════════╪════════╪════════╪═══════════╪══════════╪════════════════╪════════════════════╡
|
|
│ root@pam │ 1 │ │ │ │ │ Superuser │
|
|
└─────────────┴────────┴────────┴───────────┴──────────┴────────────────┴────────────────────┘
|
|
|
|
.. image:: images/screenshots/pbs-gui-user-management-add-user.png
|
|
:align: right
|
|
:alt: Add a new user
|
|
|
|
The superuser has full administration rights on everything, so you
|
|
normally want to add other users with less privileges. You can create a new
|
|
user with the ``user create`` subcommand or through the web interface, under
|
|
**Configuration -> User Management**. The ``create`` subcommand lets you specify
|
|
many options like ``--email`` or ``--password``. You can update or change any
|
|
user properties using the ``update`` subcommand later (**Edit** in the GUI):
|
|
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager user create john@pbs --email john@example.com
|
|
# proxmox-backup-manager user update john@pbs --firstname John --lastname Smith
|
|
# proxmox-backup-manager user update john@pbs --comment "An example user."
|
|
|
|
.. todo:: Mention how to set password without passing plaintext password as cli argument.
|
|
|
|
|
|
The resulting user list looks like this:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager user list
|
|
┌──────────┬────────┬────────┬───────────┬──────────┬──────────────────┬──────────────────┐
|
|
│ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
|
|
╞══════════╪════════╪════════╪═══════════╪══════════╪══════════════════╪══════════════════╡
|
|
│ john@pbs │ 1 │ │ John │ Smith │ john@example.com │ An example user. │
|
|
├──────────┼────────┼────────┼───────────┼──────────┼──────────────────┼──────────────────┤
|
|
│ root@pam │ 1 │ │ │ │ │ Superuser │
|
|
└──────────┴────────┴────────┴───────────┴──────────┴──────────────────┴──────────────────┘
|
|
|
|
Newly created users do not have any permissions. Please read the next
|
|
section to learn how to set access permissions.
|
|
|
|
If you want to disable a user account, you can do that by setting ``--enable`` to ``0``
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager user update john@pbs --enable 0
|
|
|
|
Or completely remove the user with:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager user remove john@pbs
|
|
|
|
|
|
Access Control
|
|
~~~~~~~~~~~~~~
|
|
|
|
By default new users do not have any permission. Instead you need to
|
|
specify what is allowed and what is not. You can do this by assigning
|
|
roles to users on specific objects like datastores or remotes. The
|
|
following roles exist:
|
|
|
|
**NoAccess**
|
|
Disable Access - nothing is allowed.
|
|
|
|
**Admin**
|
|
Can do anything.
|
|
|
|
**Audit**
|
|
Can view things, but is not allowed to change settings.
|
|
|
|
**DatastoreAdmin**
|
|
Can do anything on datastores.
|
|
|
|
**DatastoreAudit**
|
|
Can view datastore settings and list content. But
|
|
is not allowed to read the actual data.
|
|
|
|
**DatastoreReader**
|
|
Can Inspect datastore content and can do restores.
|
|
|
|
**DatastoreBackup**
|
|
Can backup and restore owned backups.
|
|
|
|
**DatastorePowerUser**
|
|
Can backup, restore, and prune owned backups.
|
|
|
|
**RemoteAdmin**
|
|
Can do anything on remotes.
|
|
|
|
**RemoteAudit**
|
|
Can view remote settings.
|
|
|
|
**RemoteSyncOperator**
|
|
Is allowed to read data from a remote.
|
|
|
|
.. image:: images/screenshots/pbs-gui-permissions-add.png
|
|
:align: right
|
|
:alt: Add permissions for user
|
|
|
|
Access permission information is stored in ``/etc/proxmox-backup/acl.cfg``. The
|
|
file contains 5 fields, separated using a colon (':') as a delimiter. A typical
|
|
entry takes the form:
|
|
|
|
``acl:1:/datastore:john@pbs:DatastoreBackup``
|
|
|
|
The data represented in each field is as follows:
|
|
|
|
#. ``acl`` identifier
|
|
#. A ``1`` or ``0``, representing whether propagation is enabled or disabled,
|
|
respectively
|
|
#. The object on which the permission is set. This can be a specific object
|
|
(single datastore, remote, etc.) or a top level object, which with
|
|
propagation enabled, represents all children of the object also.
|
|
#. The user for which the permission is set
|
|
#. The role being set
|
|
|
|
You can manage datastore permissions from **Configuration -> Permissions** in the
|
|
web interface. Likewise, you can use the ``acl`` subcommand to manage and
|
|
monitor user permissions from the command line. For example, the command below
|
|
will add the user ``john@pbs`` as a **DatastoreAdmin** for the datastore
|
|
``store1``, located at ``/backup/disk1/store1``:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager acl update /datastore/store1 DatastoreAdmin --userid john@pbs
|
|
|
|
You can monitor the roles of each user using the following command:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager acl list
|
|
┌──────────┬──────────────────┬───────────┬────────────────┐
|
|
│ ugid │ path │ propagate │ roleid │
|
|
╞══════════╪══════════════════╪═══════════╪════════════════╡
|
|
│ john@pbs │ /datastore/disk1 │ 1 │ DatastoreAdmin │
|
|
└──────────┴──────────────────┴───────────┴────────────────┘
|
|
|
|
A single user can be assigned multiple permission sets for different datastores.
|
|
|
|
.. Note::
|
|
Naming convention is important here. For datastores on the host,
|
|
you must use the convention ``/datastore/{storename}``. For example, to set
|
|
permissions for a datastore mounted at ``/mnt/backup/disk4/store2``, you would use
|
|
``/datastore/store2`` for the path. For remote stores, use the convention
|
|
``/remote/{remote}/{storename}``, where ``{remote}`` signifies the name of the
|
|
remote (see `Remote` below) and ``{storename}`` is the name of the datastore on
|
|
the remote.
|
|
|
|
Network Management
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
Proxmox Backup Server provides both a web interface and a command line tool for
|
|
network configuration. You can find the configuration options in the web
|
|
interface under the **Network Interfaces** section of the **Configuration** menu
|
|
tree item. The command line tool is accessed via the ``network`` subcommand.
|
|
These interfaces allow you to carry out some basic network management tasks,
|
|
such as adding, configuring, and removing network interfaces.
|
|
|
|
.. note:: Any changes made to the network configuration are not
|
|
applied, until you click on **Apply Configuration** or enter the ``network
|
|
reload`` command. This allows you to make many changes at once. It also allows
|
|
you to ensure that your changes are correct before applying them, as making a
|
|
mistake here can render the server inaccessible over the network.
|
|
|
|
To get a list of available interfaces, use the following command:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager network list
|
|
┌───────┬────────┬───────────┬────────┬─────────────┬──────────────┬──────────────┐
|
|
│ name │ type │ autostart │ method │ address │ gateway │ ports/slaves │
|
|
╞═══════╪════════╪═══════════╪════════╪═════════════╪══════════════╪══════════════╡
|
|
│ bond0 │ bond │ 1 │ static │ x.x.x.x/x │ x.x.x.x │ ens18 ens19 │
|
|
├───────┼────────┼───────────┼────────┼─────────────┼──────────────┼──────────────┤
|
|
│ ens18 │ eth │ 1 │ manual │ │ │ │
|
|
├───────┼────────┼───────────┼────────┼─────────────┼──────────────┼──────────────┤
|
|
│ ens19 │ eth │ 1 │ manual │ │ │ │
|
|
└───────┴────────┴───────────┴────────┴─────────────┴──────────────┴──────────────┘
|
|
|
|
.. image:: images/screenshots/pbs-gui-network-create-bond.png
|
|
:align: right
|
|
:alt: Add a network interface
|
|
|
|
To add a new network interface, use the ``create`` subcommand with the relevant
|
|
parameters. For example, you may want to set up a bond, for the purpose of
|
|
network redundancy. The following command shows a template for creating the bond shown
|
|
in the list above:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager network create bond0 --type bond --bond_mode active-backup --slaves ens18,ens19 --autostart true --cidr x.x.x.x/x --gateway x.x.x.x
|
|
|
|
You can make changes to the configuration of a network interface with the
|
|
``update`` subcommand:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager network update bond0 --cidr y.y.y.y/y
|
|
|
|
You can also remove a network interface:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager network remove bond0
|
|
|
|
The pending changes for the network configuration file will appear at the bottom of the
|
|
web interface. You can also view these changes, by using the command:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager network changes
|
|
|
|
If you would like to cancel all changes at this point, you can either click on
|
|
the **Revert** button or use the following command:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager network revert
|
|
|
|
If you are happy with the changes and would like to write them into the
|
|
configuration file, select **Apply Configuration**. The corresponding command
|
|
is:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager network reload
|
|
|
|
.. note:: This command and corresponding GUI button rely on the ``ifreload``
|
|
command, from the package ``ifupdown2``. This package is included within the
|
|
Proxmox Backup Server installation, however, you may have to install it yourself,
|
|
if you have installed Proxmox Backup Server on top of Debian or Proxmox VE.
|
|
|
|
You can also configure DNS settings, from the **DNS** section
|
|
of **Configuration** or by using the ``dns`` subcommand of
|
|
``proxmox-backup-manager``.
|
|
|
|
:term:`Remote`
|
|
~~~~~~~~~~~~~~
|
|
|
|
A remote refers to a separate Proxmox Backup Server installation and a user on that
|
|
installation, from which you can `sync` datastores to a local datastore with a
|
|
`Sync Job`. You can configure remotes in the web interface, under **Configuration
|
|
-> Remotes**. Alternatively, you can use the ``remote`` subcommand. The
|
|
configuration information for remotes is stored in the file
|
|
``/etc/proxmox-backup/remote.cfg``.
|
|
|
|
.. image:: images/screenshots/pbs-gui-remote-add.png
|
|
:align: right
|
|
:alt: Add a remote
|
|
|
|
To add a remote, you need its hostname or ip, a userid and password on the
|
|
remote, and its certificate fingerprint. To get the fingerprint, use the
|
|
``proxmox-backup-manager cert info`` command on the remote, or navigate to
|
|
**Dashboard** in the remote's web interface and select **Show Fingerprint**.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager cert info |grep Fingerprint
|
|
Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
|
|
|
|
Using the information specified above, you can add a remote from the **Remotes**
|
|
configuration panel, or by using the command:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager remote create pbs2 --host pbs2.mydomain.example --userid sync@pam --password 'SECRET' --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
|
|
|
|
Use the ``list``, ``show``, ``update``, ``remove`` subcommands of
|
|
``proxmox-backup-manager remote`` to manage your remotes:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager remote update pbs2 --host pbs2.example
|
|
# proxmox-backup-manager remote list
|
|
┌──────┬──────────────┬──────────┬───────────────────────────────────────────┬─────────┐
|
|
│ name │ host │ userid │ fingerprint │ comment │
|
|
╞══════╪══════════════╪══════════╪═══════════════════════════════════════════╪═════════╡
|
|
│ pbs2 │ pbs2.example │ sync@pam │64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe │ │
|
|
└──────┴──────────────┴──────────┴───────────────────────────────────────────┴─────────┘
|
|
# proxmox-backup-manager remote remove pbs2
|
|
|
|
.. _syncjobs:
|
|
|
|
Sync Jobs
|
|
~~~~~~~~~
|
|
|
|
.. image:: images/screenshots/pbs-gui-syncjob-add.png
|
|
:align: right
|
|
:alt: Add a Sync Job
|
|
|
|
Sync jobs are configured to pull the contents of a datastore on a **Remote** to
|
|
a local datastore. You can manage sync jobs under **Configuration -> Sync Jobs**
|
|
in the web interface, or using the ``proxmox-backup-manager sync-job`` command.
|
|
The configuration information for sync jobs is stored at
|
|
``/etc/proxmox-backup/sync.cfg``. To create a new sync job, click the add button
|
|
in the GUI, or use the ``create`` subcommand. After creating a sync job, you can
|
|
either start it manually on the GUI or provide it with a schedule (see
|
|
:ref:`calendar-events`) to run regularly.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager sync-job create pbs2-local --remote pbs2 --remote-store local --store local --schedule 'Wed 02:30'
|
|
# proxmox-backup-manager sync-job update pbs2-local --comment 'offsite'
|
|
# proxmox-backup-manager sync-job list
|
|
┌────────────┬───────┬────────┬──────────────┬───────────┬─────────┐
|
|
│ id │ store │ remote │ remote-store │ schedule │ comment │
|
|
╞════════════╪═══════╪════════╪══════════════╪═══════════╪═════════╡
|
|
│ pbs2-local │ local │ pbs2 │ local │ Wed 02:30 │ offsite │
|
|
└────────────┴───────┴────────┴──────────────┴───────────┴─────────┘
|
|
# proxmox-backup-manager sync-job remove pbs2-local
|
|
|
|
|
|
Garbage Collection
|
|
~~~~~~~~~~~~~~~~~~
|
|
You can monitor and run :ref:`garbage collection <garbage-collection>` on the
|
|
Proxmox Backup Server using the ``garbage-collection`` subcommand of
|
|
``proxmox-backup-manager``. You can use the ``start`` subcommand to manually start garbage
|
|
collection on an entire datastore and the ``status`` subcommand to see
|
|
attributes relating to the :ref:`garbage collection <garbage-collection>`.
|
|
|
|
|
|
Backup Client usage
|
|
-------------------
|
|
|
|
The command line client is called :command:`proxmox-backup-client`.
|
|
|
|
|
|
Repository Locations
|
|
~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The client uses the following notation to specify a datastore repository
|
|
on the backup server.
|
|
|
|
[[username@]server:]datastore
|
|
|
|
The default value for ``username`` ist ``root``. If no server is specified,
|
|
the default is the local host (``localhost``).
|
|
|
|
You can pass the repository with the ``--repository`` command
|
|
line option, or by setting the ``PBS_REPOSITORY`` environment
|
|
variable.
|
|
|
|
|
|
Environment Variables
|
|
~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
``PBS_REPOSITORY``
|
|
The default backup repository.
|
|
|
|
``PBS_PASSWORD``
|
|
When set, this value is used for the password required for the
|
|
backup server.
|
|
|
|
``PBS_ENCRYPTION_PASSWORD``
|
|
When set, this value is used to access the secret encryption key (if
|
|
protected by password).
|
|
|
|
``PBS_FINGERPRINT`` When set, this value is used to verify the server
|
|
certificate (only used if the system CA certificates cannot
|
|
validate the certificate).
|
|
|
|
|
|
Output Format
|
|
~~~~~~~~~~~~~
|
|
|
|
Most commands support the ``--output-format`` parameter. It accepts
|
|
the following values:
|
|
|
|
:``text``: Text format (default). Structured data is rendered as a table.
|
|
|
|
:``json``: JSON (single line).
|
|
|
|
:``json-pretty``: JSON (multiple lines, nicely formatted).
|
|
|
|
|
|
Please use the following environment variables to modify output behavior:
|
|
|
|
``PROXMOX_OUTPUT_FORMAT``
|
|
Defines the default output format.
|
|
|
|
``PROXMOX_OUTPUT_NO_BORDER``
|
|
If set (to any value), do not render table borders.
|
|
|
|
``PROXMOX_OUTPUT_NO_HEADER``
|
|
If set (to any value), do not render table headers.
|
|
|
|
.. note:: The ``text`` format is designed to be human readable, and
|
|
not meant to be parsed by automation tools. Please use the ``json``
|
|
format if you need to process the output.
|
|
|
|
|
|
.. _creating-backups:
|
|
|
|
Creating Backups
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
This section explains how to create a backup from within the machine. This can
|
|
be a physical host, a virtual machine, or a container. Such backups may contain file
|
|
and image archives. There are no restrictions in this case.
|
|
|
|
.. note:: If you want to backup virtual machines or containers on Proxmox VE, see :ref:`pve-integration`.
|
|
|
|
For the following example you need to have a backup server set up, working
|
|
credentials and need to know the repository name.
|
|
In the following examples we use ``backup-server:store1``.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client backup root.pxar:/ --repository backup-server:store1
|
|
Starting backup: host/elsa/2019-12-03T09:35:01Z
|
|
Client name: elsa
|
|
skip mount point: "/boot/efi"
|
|
skip mount point: "/dev"
|
|
skip mount point: "/run"
|
|
skip mount point: "/sys"
|
|
Uploaded 12129 chunks in 87 seconds (564 MB/s).
|
|
End Time: 2019-12-03T10:36:29+01:00
|
|
|
|
This will prompt you for a password and then uploads a file archive named
|
|
``root.pxar`` containing all the files in the ``/`` directory.
|
|
|
|
.. Caution:: Please note that the proxmox-backup-client does not
|
|
automatically include mount points. Instead, you will see a short
|
|
``skip mount point`` notice for each of them. The idea is to
|
|
create a separate file archive for each mounted disk. You can
|
|
explicitly include them using the ``--include-dev`` option
|
|
(i.e. ``--include-dev /boot/efi``). You can use this option
|
|
multiple times for each mount point that should be included.
|
|
|
|
The ``--repository`` option can get quite long and is used by all
|
|
commands. You can avoid having to enter this value by setting the
|
|
environment variable ``PBS_REPOSITORY``. Note that if you would like this to remain set
|
|
over multiple sessions, you should instead add the below line to your
|
|
``.bashrc`` file.
|
|
|
|
.. code-block:: console
|
|
|
|
# export PBS_REPOSITORY=backup-server:store1
|
|
|
|
After this you can execute all commands without specifying the ``--repository``
|
|
option.
|
|
|
|
One single backup is allowed to contain more than one archive. For example, if
|
|
you want to backup two disks mounted at ``/mmt/disk1`` and ``/mnt/disk2``:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2
|
|
|
|
This creates a backup of both disks.
|
|
|
|
The backup command takes a list of backup specifications, which
|
|
include the archive name on the server, the type of the archive, and the
|
|
archive source at the client. The format is:
|
|
|
|
<archive-name>.<type>:<source-path>
|
|
|
|
Common types are ``.pxar`` for file archives, and ``.img`` for block
|
|
device images. To create a backup of a block device run the following command:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client backup mydata.img:/dev/mylvm/mydata
|
|
|
|
Excluding files/folders from a backup
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Sometimes it is desired to exclude certain files or folders from a backup archive.
|
|
To tell the Proxmox Backup client when and how to ignore files and directories,
|
|
place a text file called ``.pxarexclude`` in the filesystem hierarchy.
|
|
Whenever the backup client encounters such a file in a directory, it interprets
|
|
each line as glob match patterns for files and directories that are to be excluded
|
|
from the backup.
|
|
|
|
The file must contain a single glob pattern per line. Empty lines are ignored.
|
|
The same is true for lines starting with ``#``, which indicates a comment.
|
|
A ``!`` at the beginning of a line reverses the glob match pattern from an exclusion
|
|
to an explicit inclusion. This makes it possible to exclude all entries in a
|
|
directory except for a few single files/subdirectories.
|
|
Lines ending in ``/`` match only on directories.
|
|
The directory containing the ``.pxarexclude`` file is considered to be the root of
|
|
the given patterns. It is only possible to match files in this directory and its subdirectories.
|
|
|
|
``\`` is used to escape special glob characters.
|
|
``?`` matches any single character.
|
|
``*`` matches any character, including an empty string.
|
|
``**`` is used to match subdirectories. It can be used to, for example, exclude
|
|
all files ending in ``.tmp`` within the directory or subdirectories with the
|
|
following pattern ``**/*.tmp``.
|
|
``[...]`` matches a single character from any of the provided characters within
|
|
the brackets. ``[!...]`` does the complementary and matches any single character
|
|
not contained within the brackets. It is also possible to specify ranges with two
|
|
characters separated by ``-``. For example, ``[a-z]`` matches any lowercase
|
|
alphabetic character and ``[0-9]`` matches any one single digit.
|
|
|
|
The order of the glob match patterns defines whether a file is included or
|
|
excluded, that is to say later entries override previous ones.
|
|
This is also true for match patterns encountered deeper down the directory tree,
|
|
which can override a previous exclusion.
|
|
Be aware that excluded directories will **not** be read by the backup client.
|
|
Thus, a ``.pxarexclude`` file in an excluded subdirectory will have no effect.
|
|
``.pxarexclude`` files are treated as regular files and will be included in the
|
|
backup archive.
|
|
|
|
For example, consider the following directory structure:
|
|
|
|
.. code-block:: console
|
|
|
|
# ls -aR folder
|
|
folder/:
|
|
. .. .pxarexclude subfolder0 subfolder1
|
|
|
|
folder/subfolder0:
|
|
. .. file0 file1 file2 file3 .pxarexclude
|
|
|
|
folder/subfolder1:
|
|
. .. file0 file1 file2 file3
|
|
|
|
The different ``.pxarexclude`` files contain the following:
|
|
|
|
.. code-block:: console
|
|
|
|
# cat folder/.pxarexclude
|
|
/subfolder0/file1
|
|
/subfolder1/*
|
|
!/subfolder1/file2
|
|
|
|
.. code-block:: console
|
|
|
|
# cat folder/subfolder0/.pxarexclude
|
|
file3
|
|
|
|
This would exclude ``file1`` and ``file3`` in ``subfolder0`` and all of
|
|
``subfolder1`` except ``file2``.
|
|
|
|
Restoring this backup will result in:
|
|
|
|
.. code-block:: console
|
|
|
|
ls -aR restored
|
|
restored/:
|
|
. .. .pxarexclude subfolder0 subfolder1
|
|
|
|
restored/subfolder0:
|
|
. .. file0 file2 .pxarexclude
|
|
|
|
restored/subfolder1:
|
|
. .. file2
|
|
|
|
Encryption
|
|
~~~~~~~~~~
|
|
|
|
Proxmox Backup supports client-side encryption with AES-256 in GCM_
|
|
mode. To set this up, you first need to create an encryption key:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client key create my-backup.key
|
|
Encryption Key Password: **************
|
|
|
|
The key is password protected by default. If you do not need this
|
|
extra protection, you can also create it without a password:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client key create /path/to/my-backup.key --kdf none
|
|
|
|
Having created this key, it is now possible to create an encrypted backup, by
|
|
passing the ``--keyfile`` parameter, with the path to the key file.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key
|
|
Password: *********
|
|
Encryption Key Password: **************
|
|
...
|
|
|
|
.. Note:: If you do not specify the name of the backup key, the key will be
|
|
created in the default location
|
|
``~/.config/proxmox-backup/encryption-key.json``. ``proxmox-backup-client``
|
|
will also search this location by default, in case the ``--keyfile``
|
|
parameter is not specified.
|
|
|
|
You can avoid entering the passwords by setting the environment
|
|
variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``.
|
|
|
|
Using a master key to store and recover encryption keys
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
You can also use ``proxmox-backup-client key`` to create an RSA public/private
|
|
key pair, which can be used to store an encrypted version of the symmetric
|
|
backup encryption key alongside each backup and recover it later.
|
|
|
|
To set up a master key:
|
|
|
|
1. Create an encryption key for the backup:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client key create
|
|
creating default key at: "~/.config/proxmox-backup/encryption-key.json"
|
|
Encryption Key Password: **********
|
|
...
|
|
|
|
The resulting file will be saved to ``~/.config/proxmox-backup/encryption-key.json``.
|
|
|
|
2. Create an RSA public/private key pair:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client key create-master-key
|
|
Master Key Password: *********
|
|
...
|
|
|
|
This will create two files in your current directory, ``master-public.pem``
|
|
and ``master-private.pem``.
|
|
|
|
3. Import the newly created ``master-public.pem`` public certificate, so that
|
|
``proxmox-backup-client`` can find and use it upon backup.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client key import-master-pubkey /path/to/master-public.pem
|
|
Imported public master key to "~/.config/proxmox-backup/master-public.pem"
|
|
|
|
4. With all these files in place, run a backup job:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client backup etc.pxar:/etc
|
|
|
|
The key will be stored in your backup, under the name ``rsa-encrypted.key``.
|
|
|
|
.. Note:: The ``--keyfile`` parameter can be excluded, if the encryption key
|
|
is in the default path. If you specified another path upon creation, you
|
|
must pass the ``--keyfile`` parameter.
|
|
|
|
5. To test that everything worked, you can restore the key from the backup:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client restore /path/to/backup/ rsa-encrypted.key /path/to/target
|
|
|
|
.. Note:: You should not need an encryption key to extract this file. However, if
|
|
a key exists at the default location
|
|
(``~/.config/proxmox-backup/encryption-key.json``) the program will prompt
|
|
you for an encryption key password. Simply moving ``encryption-key.json``
|
|
out of this directory will fix this issue.
|
|
|
|
6. Then, use the previously generated master key to decrypt the file:
|
|
|
|
.. code-block:: console
|
|
|
|
# openssl rsautl -decrypt -inkey master-private.pem -in rsa-encrypted.key -out /path/to/target
|
|
Enter pass phrase for ./master-private.pem: *********
|
|
|
|
7. The target file will now contain the encryption key information in plain
|
|
text. The success of this can be confirmed by passing the resulting ``json``
|
|
file, with the ``--keyfile`` parameter, when decrypting files from the backup.
|
|
|
|
.. warning:: Without their key, backed up files will be inaccessible. Thus, you should
|
|
keep keys ordered and in a place that is separate from the contents being
|
|
backed up. It can happen, for example, that you back up an entire system, using
|
|
a key on that system. If the system then becomes inaccessable for any reason
|
|
and needs to be restored, this will not be possible as the encryption key will be
|
|
lost along with the broken system. In preparation for the worst case scenario,
|
|
you should consider keeping a paper copy of this key locked away in
|
|
a safe place.
|
|
|
|
Restoring Data
|
|
~~~~~~~~~~~~~~
|
|
|
|
The regular creation of backups is a necessary step to avoiding data
|
|
loss. More importantly, however, is the restoration. It is good practice to perform
|
|
periodic recovery tests to ensure that you can access the data in
|
|
case of problems.
|
|
|
|
First, you need to find the snapshot which you want to restore. The snapshot
|
|
command provides a list of all the snapshots on the server:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client snapshots
|
|
┌────────────────────────────────┬─────────────┬────────────────────────────────────┐
|
|
│ snapshot │ size │ files │
|
|
╞════════════════════════════════╪═════════════╪════════════════════════════════════╡
|
|
│ host/elsa/2019-12-03T09:30:15Z │ 51788646825 │ root.pxar catalog.pcat1 index.json │
|
|
├────────────────────────────────┼─────────────┼────────────────────────────────────┤
|
|
│ host/elsa/2019-12-03T09:35:01Z │ 51790622048 │ root.pxar catalog.pcat1 index.json │
|
|
├────────────────────────────────┼─────────────┼────────────────────────────────────┤
|
|
...
|
|
|
|
You can inspect the catalog to find specific files.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client catalog dump host/elsa/2019-12-03T09:35:01Z
|
|
...
|
|
d "./root.pxar.didx/etc/cifs-utils"
|
|
l "./root.pxar.didx/etc/cifs-utils/idmap-plugin"
|
|
d "./root.pxar.didx/etc/console-setup"
|
|
...
|
|
|
|
The restore command lets you restore a single archive from the
|
|
backup.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/
|
|
|
|
To get the contents of any archive, you can restore the ``index.json`` file in the
|
|
repository to the target path '-'. This will dump the contents to the standard output.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json -
|
|
|
|
|
|
Interactive Restores
|
|
^^^^^^^^^^^^^^^^^^^^
|
|
|
|
If you only want to restore a few individual files, it is often easier
|
|
to use the interactive recovery shell.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client catalog shell host/elsa/2019-12-03T09:35:01Z root.pxar
|
|
Starting interactive shell
|
|
pxar:/ > ls
|
|
bin boot dev etc home lib lib32
|
|
...
|
|
|
|
The interactive recovery shell is a minimalistic command line interface that
|
|
utilizes the metadata stored in the catalog to quickly list, navigate and
|
|
search files in a file archive.
|
|
To restore files, you can select them individually or match them with a glob
|
|
pattern.
|
|
|
|
Using the catalog for navigation reduces the overhead considerably because only
|
|
the catalog needs to be downloaded and, optionally, decrypted.
|
|
The actual chunks are only accessed if the metadata in the catalog is not enough
|
|
or for the actual restore.
|
|
|
|
Similar to common UNIX shells ``cd`` and ``ls`` are the commands used to change
|
|
working directory and list directory contents in the archive.
|
|
``pwd`` shows the full path of the current working directory with respect to the
|
|
archive root.
|
|
|
|
Being able to quickly search the contents of the archive is a commmonly needed feature.
|
|
That's where the catalog is most valuable.
|
|
For example:
|
|
|
|
.. code-block:: console
|
|
|
|
pxar:/ > find etc/**/*.txt --select
|
|
"/etc/X11/rgb.txt"
|
|
pxar:/ > list-selected
|
|
etc/**/*.txt
|
|
pxar:/ > restore-selected /target/path
|
|
...
|
|
|
|
This will find and print all files ending in ``.txt`` located in ``etc/`` or a
|
|
subdirectory and add the corresponding pattern to the list for subsequent restores.
|
|
``list-selected`` shows these patterns and ``restore-selected`` finally restores
|
|
all files in the archive matching the patterns to ``/target/path`` on the local
|
|
host. This will scan the whole archive.
|
|
|
|
With ``restore /target/path`` you can restore the sub-archive given by the current
|
|
working directory to the local target path ``/target/path`` on your host.
|
|
By additionally passing a glob pattern with ``--pattern <glob>``, the restore is
|
|
further limited to files matching the pattern.
|
|
For example:
|
|
|
|
.. code-block:: console
|
|
|
|
pxar:/ > cd /etc/
|
|
pxar:/etc/ > restore /target/ --pattern **/*.conf
|
|
...
|
|
|
|
The above will scan trough all the directories below ``/etc`` and restore all
|
|
files ending in ``.conf``.
|
|
|
|
.. todo:: Explain interactive restore in more detail
|
|
|
|
Mounting of Archives via FUSE
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
The :term:`FUSE` implementation for the pxar archive allows you to mount a
|
|
file archive as a read-only filesystem to a mountpoint on your host.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client mount host/backup-client/2020-01-29T11:29:22Z root.pxar /mnt/mountpoint
|
|
# ls /mnt/mountpoint
|
|
bin dev home lib32 libx32 media opt root sbin sys usr
|
|
boot etc lib lib64 lost+found mnt proc run srv tmp var
|
|
|
|
This allows you to access the full contents of the archive in a seamless manner.
|
|
|
|
.. note:: As the FUSE connection needs to fetch and decrypt chunks from the
|
|
backup server's datastore, this can cause some additional network and CPU
|
|
load on your host, depending on the operations you perform on the mounted
|
|
filesystem.
|
|
|
|
To unmount the filesystem use the ``umount`` command on the mountpoint:
|
|
|
|
.. code-block:: console
|
|
|
|
# umount /mnt/mountpoint
|
|
|
|
Login and Logout
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
The client tool prompts you to enter the logon password as soon as you
|
|
want to access the backup server. The server checks your credentials
|
|
and responds with a ticket that is valid for two hours. The client
|
|
tool automatically stores that ticket and uses it for further requests
|
|
to this server.
|
|
|
|
You can also manually trigger this login/logout using the login and
|
|
logout commands:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client login
|
|
Password: **********
|
|
|
|
To remove the ticket, issue a logout:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client logout
|
|
|
|
|
|
.. _pruning:
|
|
|
|
Pruning and Removing Backups
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
You can manually delete a backup snapshot using the ``forget``
|
|
command:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client forget <snapshot>
|
|
|
|
|
|
.. caution:: This command removes all archives in this backup
|
|
snapshot. They will be inaccessible and unrecoverable.
|
|
|
|
|
|
Although manual removal is sometimes required, the ``prune``
|
|
command is normally used to systematically delete older backups. Prune lets
|
|
you specify which backup snapshots you want to keep. The
|
|
following retention options are available:
|
|
|
|
``--keep-last <N>``
|
|
Keep the last ``<N>`` backup snapshots.
|
|
|
|
``--keep-hourly <N>``
|
|
Keep backups for the last ``<N>`` hours. If there is more than one
|
|
backup for a single hour, only the latest is kept.
|
|
|
|
``--keep-daily <N>``
|
|
Keep backups for the last ``<N>`` days. If there is more than one
|
|
backup for a single day, only the latest is kept.
|
|
|
|
``--keep-weekly <N>``
|
|
Keep backups for the last ``<N>`` weeks. If there is more than one
|
|
backup for a single week, only the latest is kept.
|
|
|
|
.. note:: Weeks start on Monday and end on Sunday. The software
|
|
uses the `ISO week date`_ system and handles weeks at
|
|
the end of the year correctly.
|
|
|
|
``--keep-monthly <N>``
|
|
Keep backups for the last ``<N>`` months. If there is more than one
|
|
backup for a single month, only the latest is kept.
|
|
|
|
``--keep-yearly <N>``
|
|
Keep backups for the last ``<N>`` years. If there is more than one
|
|
backup for a single year, only the latest is kept.
|
|
|
|
The retention options are processed in the order given above. Each option
|
|
only covers backups within its time period. The next option does not take care
|
|
of already covered backups. It will only consider older backups.
|
|
|
|
Unfinished and incomplete backups will be removed by the prune command unless
|
|
they are newer than the last successful backup. In this case, the last failed
|
|
backup is retained.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client prune <group> --keep-daily 7 --keep-weekly 4 --keep-monthly 3
|
|
|
|
|
|
You can use the ``--dry-run`` option to test your settings. This only
|
|
shows the list of existing snapshots and what actions prune would take.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3
|
|
┌────────────────────────────────┬──────┐
|
|
│ snapshot │ keep │
|
|
╞════════════════════════════════╪══════╡
|
|
│ host/elsa/2019-12-04T13:20:37Z │ 1 │
|
|
├────────────────────────────────┼──────┤
|
|
│ host/elsa/2019-12-03T09:35:01Z │ 0 │
|
|
├────────────────────────────────┼──────┤
|
|
│ host/elsa/2019-11-22T11:54:47Z │ 1 │
|
|
├────────────────────────────────┼──────┤
|
|
│ host/elsa/2019-11-21T12:36:25Z │ 0 │
|
|
├────────────────────────────────┼──────┤
|
|
│ host/elsa/2019-11-10T10:42:20Z │ 1 │
|
|
└────────────────────────────────┴──────┘
|
|
|
|
.. note:: Neither the ``prune`` command nor the ``forget`` command free space
|
|
in the chunk-store. The chunk-store still contains the data blocks. To free
|
|
space you need to perform :ref:`garbage-collection`.
|
|
|
|
|
|
.. _garbage-collection:
|
|
|
|
Garbage Collection
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
The ``prune`` command removes only the backup index files, not the data
|
|
from the datastore. This task is left to the garbage collection
|
|
command. It is recommended to carry out garbage collection on a regular basis.
|
|
|
|
The garbage collection works in two phases. In the first phase, all
|
|
data blocks that are still in use are marked. In the second phase,
|
|
unused data blocks are removed.
|
|
|
|
.. note:: This command needs to read all existing backup index files
|
|
and touches the complete chunk-store. This can take a long time
|
|
depending on the number of chunks and the speed of the underlying
|
|
disks.
|
|
|
|
.. note:: The garbage collection will only remove chunks that haven't been used
|
|
for at least one day (exactly 24h 5m). This grace period is necessary because
|
|
chunks in use are marked by touching the chunk which updates the ``atime``
|
|
(access time) property. Filesystems are mounted with the ``relatime`` option
|
|
by default. This results in a better performance by only updating the
|
|
``atime`` property if the last access has been at least 24 hours ago. The
|
|
downside is, that touching a chunk within these 24 hours will not always
|
|
update its ``atime`` property.
|
|
|
|
Chunks in the grace period will be logged at the end of the garbage
|
|
collection task as *Pending removals*.
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client garbage-collect
|
|
starting garbage collection on store store2
|
|
Start GC phase1 (mark used chunks)
|
|
Start GC phase2 (sweep unused chunks)
|
|
percentage done: 1, chunk count: 219
|
|
percentage done: 2, chunk count: 453
|
|
...
|
|
percentage done: 99, chunk count: 21188
|
|
Removed bytes: 411368505
|
|
Removed chunks: 203
|
|
Original data bytes: 327160886391
|
|
Disk bytes: 52767414743 (16 %)
|
|
Disk chunks: 21221
|
|
Average chunk size: 2486565
|
|
TASK OK
|
|
|
|
|
|
.. todo:: howto run garbage-collection at regular intervalls (cron)
|
|
|
|
Benchmarking
|
|
~~~~~~~~~~~~
|
|
The backup client also comes with a benchmarking tool. This tool measures
|
|
various metrics relating to compression and encryption speeds. You can run a
|
|
benchmark using the ``benchmark`` subcommand of ``proxmox-backup-client``:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-client benchmark
|
|
Uploaded 656 chunks in 5 seconds.
|
|
Time per request: 7659 microseconds.
|
|
TLS speed: 547.60 MB/s
|
|
SHA256 speed: 585.76 MB/s
|
|
Compression speed: 1923.96 MB/s
|
|
Decompress speed: 7885.24 MB/s
|
|
AES256/GCM speed: 3974.03 MB/s
|
|
┌───────────────────────────────────┬─────────────────────┐
|
|
│ Name │ Value │
|
|
╞═══════════════════════════════════╪═════════════════════╡
|
|
│ TLS (maximal backup upload speed) │ 547.60 MB/s (93%) │
|
|
├───────────────────────────────────┼─────────────────────┤
|
|
│ SHA256 checksum computation speed │ 585.76 MB/s (28%) │
|
|
├───────────────────────────────────┼─────────────────────┤
|
|
│ ZStd level 1 compression speed │ 1923.96 MB/s (89%) │
|
|
├───────────────────────────────────┼─────────────────────┤
|
|
│ ZStd level 1 decompression speed │ 7885.24 MB/s (98%) │
|
|
├───────────────────────────────────┼─────────────────────┤
|
|
│ AES256 GCM encryption speed │ 3974.03 MB/s (104%) │
|
|
└───────────────────────────────────┴─────────────────────┘
|
|
|
|
.. note:: The percentages given in the output table correspond to a
|
|
comparison against a Ryzen 7 2700X. The TLS test connects to the
|
|
local host, so there is no network involved.
|
|
|
|
You can also pass the ``--output-format`` parameter to output stats in ``json``,
|
|
rather than the default table format.
|
|
|
|
.. _pve-integration:
|
|
|
|
`Proxmox VE`_ integration
|
|
-------------------------
|
|
|
|
You need to define a new storage with type 'pbs' on your `Proxmox VE`_
|
|
node. The following example uses ``store2`` as storage name, and
|
|
assumes the server address is ``localhost``, and you want to connect
|
|
as ``user1@pbs``.
|
|
|
|
.. code-block:: console
|
|
|
|
# pvesm add pbs store2 --server localhost --datastore store2
|
|
# pvesm set store2 --username user1@pbs --password <secret>
|
|
|
|
.. note:: If you would rather not pass your password as plain text, you can pass
|
|
the ``--password`` parameter, without any arguments. This will cause the
|
|
program to prompt you for a password upon entering the command.
|
|
|
|
If your backup server uses a self signed certificate, you need to add
|
|
the certificate fingerprint to the configuration. You can get the
|
|
fingerprint by running the following command on the backup server:
|
|
|
|
.. code-block:: console
|
|
|
|
# proxmox-backup-manager cert info | grep Fingerprint
|
|
Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
|
|
|
|
Please add that fingerprint to your configuration to establish a trust
|
|
relationship:
|
|
|
|
.. code-block:: console
|
|
|
|
# pvesm set store2 --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
|
|
|
|
After that you should be able to see storage status with:
|
|
|
|
.. code-block:: console
|
|
|
|
# pvesm status --storage store2
|
|
Name Type Status Total Used Available %
|
|
store2 pbs active 3905109820 1336687816 2568422004 34.23%
|
|
|
|
Having added the PBS datastore to `Proxmox VE`_, you can backup VMs and
|
|
containers in the same way you would for any other storage device within the
|
|
environment (see `PVE Admin Guide: Backup and Restore
|
|
<https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_vzdump>`_.
|
|
|
|
|
|
.. include:: command-line-tools.rst
|
|
|
|
.. include:: services.rst
|