2020-12-19 10:14:56 +00:00
|
|
|
Tape Backup
|
|
|
|
===========
|
|
|
|
|
2021-01-25 15:30:54 +00:00
|
|
|
Proxmox tape backup provides an easy way to store datastore content
|
|
|
|
onto magnetic tapes. This increases data safety because you get:
|
2020-12-19 10:14:56 +00:00
|
|
|
|
|
|
|
- an additional copy of the data
|
|
|
|
- to a different media type (tape)
|
2021-01-12 15:37:23 +00:00
|
|
|
- to an additional location (you can move tapes offsite)
|
2020-12-19 10:14:56 +00:00
|
|
|
|
2021-01-25 15:30:54 +00:00
|
|
|
In most restore jobs, only data from the last backup job is restored.
|
|
|
|
Restore requests further decline the older the data
|
2021-01-12 15:37:23 +00:00
|
|
|
gets. Considering this, tape backup may also help to reduce disk
|
|
|
|
usage, because you can safely remove data from disk once archived on
|
|
|
|
tape. This is especially true if you need to keep data for several
|
|
|
|
years.
|
2020-12-21 11:13:35 +00:00
|
|
|
|
|
|
|
Tape backups do not provide random access to the stored data. Instead,
|
|
|
|
you need to restore the data to disk before you can access it
|
|
|
|
again. Also, if you store your tapes offsite (using some kind of tape
|
|
|
|
vaulting service), you need to bring them onsite before you can do any
|
|
|
|
restore. So please consider that restores from tapes can take much
|
|
|
|
longer than restores from disk.
|
2020-12-19 10:14:56 +00:00
|
|
|
|
|
|
|
|
|
|
|
Tape Technology Primer
|
|
|
|
----------------------
|
|
|
|
|
|
|
|
.. _Linear Tape Open: https://en.wikipedia.org/wiki/Linear_Tape-Open
|
|
|
|
|
|
|
|
As of 2021, the only broadly available tape technology standard is
|
|
|
|
`Linear Tape Open`_, and different vendors offers LTO Ultrium tape
|
2020-12-21 11:13:35 +00:00
|
|
|
drives, autoloaders and LTO tape cartridges.
|
|
|
|
|
2021-01-25 15:30:54 +00:00
|
|
|
There are a few vendors offering proprietary drives with
|
2020-12-21 11:13:35 +00:00
|
|
|
slight advantages in performance and capacity, but they have
|
2021-01-25 15:30:54 +00:00
|
|
|
significant disadvantages:
|
2020-12-21 11:13:35 +00:00
|
|
|
|
|
|
|
- proprietary (single vendor)
|
|
|
|
- a much higher purchase cost
|
|
|
|
|
2021-01-25 15:30:54 +00:00
|
|
|
So we currently do not test such drives.
|
2020-12-19 10:14:56 +00:00
|
|
|
|
|
|
|
In general, LTO tapes offer the following advantages:
|
|
|
|
|
|
|
|
- Durable (30 years)
|
|
|
|
- High Capacity (12 TB)
|
|
|
|
- Relatively low cost per TB
|
|
|
|
- Cold Media
|
|
|
|
- Movable (storable inside vault)
|
|
|
|
- Multiple vendors (for both media and drives)
|
2021-01-23 14:19:28 +00:00
|
|
|
- Build in AES-CGM Encryption engine
|
2020-12-19 14:13:38 +00:00
|
|
|
|
2021-01-06 08:45:36 +00:00
|
|
|
Please note that `Proxmox Backup Server` already stores compressed
|
2021-01-12 15:37:23 +00:00
|
|
|
data, so we do not need/use the tape compression feature.
|
2020-12-21 11:13:35 +00:00
|
|
|
|
2020-12-19 14:13:38 +00:00
|
|
|
|
|
|
|
Supported Hardware
|
|
|
|
------------------
|
|
|
|
|
2021-01-23 14:19:28 +00:00
|
|
|
Proxmox Backup Server supports `Linear Tape Open`_ genertion 4 (LTO4)
|
|
|
|
or later. In general, all SCSI2 tape drives supported by the Linux
|
|
|
|
kernel should work, but feature like hardware encryptions needs LTO4
|
|
|
|
or later.
|
2020-12-19 14:13:38 +00:00
|
|
|
|
|
|
|
Tape changer support is done using the Linux 'mtx' command line
|
2021-01-25 15:30:54 +00:00
|
|
|
tool. So any changer device supported by that tool should work.
|
2020-12-19 14:13:38 +00:00
|
|
|
|
|
|
|
|
2020-12-19 15:56:54 +00:00
|
|
|
Drive Performance
|
|
|
|
~~~~~~~~~~~~~~~~~
|
|
|
|
|
2021-01-25 15:30:54 +00:00
|
|
|
Current LTO-8 tapes provide read/write speeds up to 360MB/s. This means,
|
|
|
|
that it still takes a minimum of 9 hours to completely write or
|
2020-12-19 15:56:54 +00:00
|
|
|
read a single tape (even at maximum speed).
|
|
|
|
|
|
|
|
The only way to speed up that data rate is to use more than one
|
|
|
|
drive. That way you can run several backup jobs in parallel, or run
|
|
|
|
restore jobs while the other dives are used for backups.
|
|
|
|
|
2020-12-20 08:16:09 +00:00
|
|
|
Also consider that you need to read data first from your datastore
|
|
|
|
(disk). But a single spinning disk is unable to deliver data at this
|
2021-01-25 15:30:54 +00:00
|
|
|
rate. We measured a maximum rate of about 60MB/s to 100MB/s in practice,
|
|
|
|
so it takes 33 hours to read 12TB to fill up an LTO-8 tape. If you want
|
2021-01-12 15:37:23 +00:00
|
|
|
to run your tape at full speed, please make sure that the source
|
2021-01-25 15:30:54 +00:00
|
|
|
datastore is able to deliver that performance (e.g, by using SSDs).
|
2020-12-20 08:16:09 +00:00
|
|
|
|
2020-12-19 14:13:38 +00:00
|
|
|
|
|
|
|
Terminology
|
|
|
|
-----------
|
|
|
|
|
|
|
|
:Tape Labels: are used to uniquely indentify a tape. You normally use
|
|
|
|
some sticky paper labels and apply them on the front of the
|
|
|
|
cartridge. We additionally store the label text magnetically on the
|
|
|
|
tape (first file on tape).
|
|
|
|
|
|
|
|
.. _Code 39: https://en.wikipedia.org/wiki/Code_39
|
|
|
|
|
|
|
|
.. _LTO Ultrium Cartridge Label Specification: https://www.ibm.com/support/pages/ibm-lto-ultrium-cartridge-label-specification
|
|
|
|
|
2020-12-20 09:41:40 +00:00
|
|
|
.. _LTO Barcode Generator: lto-barcode/index.html
|
|
|
|
|
2020-12-19 14:13:38 +00:00
|
|
|
:Barcodes: are a special form of tape labels, which are electronically
|
|
|
|
readable. Most LTO tape robots use an 8 character string encoded as
|
|
|
|
`Code 39`_, as definded in the `LTO Ultrium Cartridge Label
|
|
|
|
Specification`_.
|
|
|
|
|
2021-01-25 15:30:54 +00:00
|
|
|
You can either buy such barcode labels from your cartridge vendor,
|
2020-12-20 09:41:40 +00:00
|
|
|
or print them yourself. You can use our `LTO Barcode Generator`_ App
|
|
|
|
for that.
|
2020-12-19 14:13:38 +00:00
|
|
|
|
|
|
|
.. Note:: Physical labels and the associated adhesive shall have an
|
|
|
|
environmental performance to match or exceed the environmental
|
|
|
|
specifications of the cartridge to which it is applied.
|
|
|
|
|
|
|
|
:Media Pools: A media pool is a logical container for tapes. A backup
|
|
|
|
job targets one media pool, so a job only uses tapes from that
|
2021-01-08 07:46:25 +00:00
|
|
|
pool. The pool additionally defines how long a backup job can
|
|
|
|
append data to tapes (allocation policy) and how long you want to
|
|
|
|
keep the data (retention policy).
|
|
|
|
|
|
|
|
:Media Set: A group of continuously written tapes (all from the same
|
|
|
|
media pool).
|
2020-12-19 14:13:38 +00:00
|
|
|
|
|
|
|
:Tape drive: The decive used to read and write data to the tape. There
|
|
|
|
are standalone drives, but drives often ship within tape libraries.
|
|
|
|
|
|
|
|
:Tape changer: A device which can change the tapes inside a tape drive
|
|
|
|
(tape robot). They are usually part of a tape library.
|
|
|
|
|
|
|
|
.. _Tape Library: https://en.wikipedia.org/wiki/Tape_library
|
|
|
|
|
|
|
|
:`Tape library`_: A storage device that contains one or more tape drives,
|
|
|
|
a number of slots to hold tape cartridges, a barcode reader to
|
|
|
|
identify tape cartridges and an automated method for loading tapes
|
|
|
|
(a robot).
|
|
|
|
|
|
|
|
People als call this 'autoloader', 'tape robot' or 'tape jukebox'.
|
2020-12-19 15:56:54 +00:00
|
|
|
|
2021-01-15 08:44:20 +00:00
|
|
|
:Inventory: The inventory stores the list of known tapes (with
|
|
|
|
additional status information).
|
|
|
|
|
|
|
|
:Catalog: A media catalog stores information about the media content.
|
|
|
|
|
2020-12-19 15:56:54 +00:00
|
|
|
|
|
|
|
Tape Quickstart
|
|
|
|
---------------
|
|
|
|
|
|
|
|
1. Configure your tape hardware (drives and changers)
|
|
|
|
|
|
|
|
2. Configure one or more media pools
|
|
|
|
|
|
|
|
3. Label your tape cartridges.
|
|
|
|
|
|
|
|
4. Start your first tape backup job ...
|
2020-12-20 08:16:09 +00:00
|
|
|
|
|
|
|
|
|
|
|
Configuration
|
|
|
|
-------------
|
|
|
|
|
|
|
|
Please note that you can configure anything using the graphical user
|
|
|
|
interface or the command line interface. Both methods results in the
|
|
|
|
same configuration.
|
|
|
|
|
|
|
|
|
|
|
|
Tape changers
|
|
|
|
~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
Tape changers (robots) are part of a `Tape Library`_. You can skip
|
|
|
|
this step if you are using a standalone drive.
|
|
|
|
|
|
|
|
Linux is able to auto detect those devices, and you can get a list
|
2021-01-06 08:45:36 +00:00
|
|
|
of available devices using::
|
2020-12-20 08:16:09 +00:00
|
|
|
|
|
|
|
# proxmox-tape changer scan
|
2021-01-06 08:45:36 +00:00
|
|
|
┌─────────────────────────────┬─────────┬──────────────┬────────┐
|
|
|
|
│ path │ vendor │ model │ serial │
|
|
|
|
╞═════════════════════════════╪═════════╪══════════════╪════════╡
|
|
|
|
│ /dev/tape/by-id/scsi-CC2C52 │ Quantum │ Superloader3 │ CC2C52 │
|
|
|
|
└─────────────────────────────┴─────────┴──────────────┴────────┘
|
2020-12-20 08:16:09 +00:00
|
|
|
|
|
|
|
In order to use that device with Proxmox, you need to create a
|
2021-01-06 15:00:31 +00:00
|
|
|
configuration entry::
|
2020-12-20 08:16:09 +00:00
|
|
|
|
2021-01-06 08:45:36 +00:00
|
|
|
# proxmox-tape changer create sl3 --path /dev/tape/by-id/scsi-CC2C52
|
2020-12-20 08:16:09 +00:00
|
|
|
|
|
|
|
Where ``sl3`` is an arbitrary name you can choose.
|
|
|
|
|
2021-01-06 15:00:31 +00:00
|
|
|
.. Note:: Please use stable device path names from inside
|
2020-12-20 08:16:09 +00:00
|
|
|
``/dev/tape/by-id/``. Names like ``/dev/sg0`` may point to a
|
|
|
|
different device after reboot, and that is not what you want.
|
|
|
|
|
2021-01-06 08:45:36 +00:00
|
|
|
You can show the final configuration with::
|
2020-12-20 08:16:09 +00:00
|
|
|
|
2021-01-06 15:00:31 +00:00
|
|
|
# proxmox-tape changer config sl3
|
|
|
|
┌──────┬─────────────────────────────┐
|
|
|
|
│ Name │ Value │
|
|
|
|
╞══════╪═════════════════════════════╡
|
|
|
|
│ name │ sl3 │
|
|
|
|
├──────┼─────────────────────────────┤
|
|
|
|
│ path │ /dev/tape/by-id/scsi-CC2C52 │
|
|
|
|
└──────┴─────────────────────────────┘
|
|
|
|
|
|
|
|
Or simply list all configured changer devices::
|
|
|
|
|
2020-12-20 08:16:09 +00:00
|
|
|
# proxmox-tape changer list
|
2021-01-06 08:45:36 +00:00
|
|
|
┌──────┬─────────────────────────────┬─────────┬──────────────┬────────────┐
|
|
|
|
│ name │ path │ vendor │ model │ serial │
|
|
|
|
╞══════╪═════════════════════════════╪═════════╪══════════════╪════════════╡
|
|
|
|
│ sl3 │ /dev/tape/by-id/scsi-CC2C52 │ Quantum │ Superloader3 │ CC2C52 │
|
|
|
|
└──────┴─────────────────────────────┴─────────┴──────────────┴────────────┘
|
2020-12-20 08:16:09 +00:00
|
|
|
|
|
|
|
The Vendor, Model and Serial number are auto detected, but only shown
|
|
|
|
if the device is online.
|
|
|
|
|
2021-01-06 08:45:36 +00:00
|
|
|
To test your setup, please query the status of the changer device with::
|
|
|
|
|
|
|
|
# proxmox-tape changer status sl3
|
|
|
|
┌───────────────┬──────────┬────────────┬─────────────┐
|
|
|
|
│ entry-kind │ entry-id │ changer-id │ loaded-slot │
|
|
|
|
╞═══════════════╪══════════╪════════════╪═════════════╡
|
|
|
|
│ drive │ 0 │ vtape1 │ 1 │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ slot │ 1 │ │ │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ slot │ 2 │ vtape2 │ │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ ... │ ... │ │ │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ slot │ 16 │ │ │
|
|
|
|
└───────────────┴──────────┴────────────┴─────────────┘
|
|
|
|
|
2021-01-06 13:11:35 +00:00
|
|
|
Tape libraries usually provide some special import/export slots (also
|
|
|
|
called "mail slots"). Tapes inside those slots are acessible from
|
|
|
|
outside, making it easy to add/remove tapes to/from the library. Those
|
|
|
|
tapes are considered to be "offline", so backup jobs will not use
|
|
|
|
them. Those special slots are auto-detected and marked as
|
|
|
|
``import-export`` slot in the status command.
|
|
|
|
|
|
|
|
It's worth noting that some of the smaller tape libraries don't have
|
|
|
|
such slots. While they have something called "Mail Slot", that slot
|
|
|
|
is just a way to grab the tape from the gripper. But they are unable
|
|
|
|
to hold media while the robot does other things. They also do not
|
|
|
|
expose that "Mail Slot" over the SCSI interface, so you wont see them in
|
|
|
|
the status output.
|
|
|
|
|
|
|
|
As a workaround, you can mark some of the normal slots as export
|
|
|
|
slot. The software treats those slots like real ``import-export``
|
|
|
|
slots, and the media inside those slots is considered to be 'offline'
|
|
|
|
(not available for backup)::
|
|
|
|
|
|
|
|
# proxmox-tape changer update sl3 --export-slots 15,16
|
|
|
|
|
|
|
|
After that, you can see those artificial ``import-export`` slots in
|
|
|
|
the status output::
|
|
|
|
|
2021-01-06 15:00:31 +00:00
|
|
|
# proxmox-tape changer status sl3
|
2021-01-06 13:11:35 +00:00
|
|
|
┌───────────────┬──────────┬────────────┬─────────────┐
|
|
|
|
│ entry-kind │ entry-id │ changer-id │ loaded-slot │
|
|
|
|
╞═══════════════╪══════════╪════════════╪═════════════╡
|
|
|
|
│ drive │ 0 │ vtape1 │ 1 │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ import-export │ 15 │ │ │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ import-export │ 16 │ │ │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ slot │ 1 │ │ │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ slot │ 2 │ vtape2 │ │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ ... │ ... │ │ │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ slot │ 14 │ │ │
|
|
|
|
└───────────────┴──────────┴────────────┴─────────────┘
|
2021-01-06 08:45:36 +00:00
|
|
|
|
2020-12-20 08:16:09 +00:00
|
|
|
|
|
|
|
Tape drives
|
|
|
|
~~~~~~~~~~~
|
|
|
|
|
2021-01-06 15:00:31 +00:00
|
|
|
Linux is able to auto detect tape drives, and you can get a list
|
|
|
|
of available tape drives using::
|
|
|
|
|
|
|
|
# proxmox-tape drive scan
|
|
|
|
┌────────────────────────────────┬────────┬─────────────┬────────┐
|
|
|
|
│ path │ vendor │ model │ serial │
|
|
|
|
╞════════════════════════════════╪════════╪═════════════╪════════╡
|
|
|
|
│ /dev/tape/by-id/scsi-12345-nst │ IBM │ ULT3580-TD4 │ 12345 │
|
|
|
|
└────────────────────────────────┴────────┴─────────────┴────────┘
|
|
|
|
|
|
|
|
In order to use that drive with Proxmox, you need to create a
|
|
|
|
configuration entry::
|
|
|
|
|
|
|
|
# proxmox-tape drive create mydrive --path /dev/tape/by-id/scsi-12345-nst
|
|
|
|
|
|
|
|
.. Note:: Please use stable device path names from inside
|
|
|
|
``/dev/tape/by-id/``. Names like ``/dev/nst0`` may point to a
|
|
|
|
different device after reboot, and that is not what you want.
|
|
|
|
|
|
|
|
If you have a tape library, you also need to set the associated
|
|
|
|
changer device::
|
|
|
|
|
2021-01-28 11:59:55 +00:00
|
|
|
# proxmox-tape drive update mydrive --changer sl3 --changer-drivenum 0
|
2021-01-06 15:00:31 +00:00
|
|
|
|
2021-01-28 11:59:55 +00:00
|
|
|
The ``--changer-drivenum`` is only necessary if the tape library
|
2021-01-06 15:00:31 +00:00
|
|
|
includes more than one drive (The changer status command lists all
|
2021-01-28 11:59:55 +00:00
|
|
|
drivenums).
|
2021-01-06 15:00:31 +00:00
|
|
|
|
|
|
|
You can show the final configuration with::
|
|
|
|
|
|
|
|
# proxmox-tape drive config mydrive
|
|
|
|
┌─────────┬────────────────────────────────┐
|
|
|
|
│ Name │ Value │
|
|
|
|
╞═════════╪════════════════════════════════╡
|
|
|
|
│ name │ mydrive │
|
|
|
|
├─────────┼────────────────────────────────┤
|
|
|
|
│ path │ /dev/tape/by-id/scsi-12345-nst │
|
|
|
|
├─────────┼────────────────────────────────┤
|
|
|
|
│ changer │ sl3 │
|
|
|
|
└─────────┴────────────────────────────────┘
|
|
|
|
|
2021-01-28 11:59:55 +00:00
|
|
|
.. NOTE:: The ``changer-drivenum`` value 0 is not stored in the
|
2021-01-06 15:00:31 +00:00
|
|
|
configuration, because that is the default.
|
|
|
|
|
|
|
|
To list all configured drives use::
|
|
|
|
|
|
|
|
# proxmox-tape drive list
|
|
|
|
┌──────────┬────────────────────────────────┬─────────┬────────┬─────────────┬────────┐
|
|
|
|
│ name │ path │ changer │ vendor │ model │ serial │
|
|
|
|
╞══════════╪════════════════════════════════╪═════════╪════════╪═════════════╪════════╡
|
|
|
|
│ mydrive │ /dev/tape/by-id/scsi-12345-nst │ sl3 │ IBM │ ULT3580-TD4 │ 12345 │
|
|
|
|
└──────────┴────────────────────────────────┴─────────┴────────┴─────────────┴────────┘
|
|
|
|
|
|
|
|
The Vendor, Model and Serial number are auto detected, but only shown
|
|
|
|
if the device is online.
|
|
|
|
|
2021-01-06 15:13:58 +00:00
|
|
|
For testing, you can simply query the drive status with::
|
|
|
|
|
|
|
|
# proxmox-tape status --drive mydrive
|
|
|
|
┌───────────┬────────────────────────┐
|
|
|
|
│ Name │ Value │
|
|
|
|
╞═══════════╪════════════════════════╡
|
|
|
|
│ blocksize │ 0 │
|
|
|
|
├───────────┼────────────────────────┤
|
|
|
|
│ status │ DRIVE_OPEN | IM_REP_EN │
|
|
|
|
└───────────┴────────────────────────┘
|
|
|
|
|
|
|
|
.. NOTE:: Blocksize should always be 0 (variable block size
|
|
|
|
mode). This is the default anyways.
|
|
|
|
|
2020-12-20 08:16:09 +00:00
|
|
|
|
|
|
|
Media Pools
|
|
|
|
~~~~~~~~~~~
|
|
|
|
|
2021-01-08 07:46:25 +00:00
|
|
|
A media pool is a logical container for tapes. A backup job targets
|
|
|
|
one media pool, so a job only uses tapes from that pool.
|
|
|
|
|
|
|
|
.. topic:: Media Set
|
|
|
|
|
|
|
|
A media set is a group of continuously written tapes, used to split
|
|
|
|
the larger pool into smaller, restorable units. One or more backup
|
2021-01-08 15:53:46 +00:00
|
|
|
jobs write to a media set, producing an ordered group of
|
|
|
|
tapes. Media sets are identified by an unique ID. That ID and the
|
|
|
|
sequence number is stored on each tape of that set (tape label).
|
2021-01-08 07:46:25 +00:00
|
|
|
|
|
|
|
Media sets are the basic unit for restore tasks, i.e. you need all
|
2021-01-08 15:53:46 +00:00
|
|
|
tapes in the set to restore the media set content. Data is fully
|
2021-01-08 07:46:25 +00:00
|
|
|
deduplicated inside a media set.
|
|
|
|
|
|
|
|
|
|
|
|
.. topic:: Media Set Allocation Policy
|
|
|
|
|
|
|
|
The pool additionally defines how long backup jobs can append data
|
2021-01-08 15:53:46 +00:00
|
|
|
to a media set. The following settings are possible:
|
2021-01-08 07:46:25 +00:00
|
|
|
|
2021-01-08 15:53:46 +00:00
|
|
|
- Try to use the current media set.
|
2021-01-08 07:46:25 +00:00
|
|
|
|
2021-01-08 15:53:46 +00:00
|
|
|
This setting produce one large media set. While this is very
|
|
|
|
space efficient (deduplication, no unused space), it can lead to
|
|
|
|
long restore times, because restore jobs needs to read all tapes in the
|
|
|
|
set.
|
2021-01-08 07:46:25 +00:00
|
|
|
|
2021-01-08 15:53:46 +00:00
|
|
|
.. NOTE:: Data is fully deduplicated inside a media set. That
|
|
|
|
also means that data is randomly distributed over the tapes in
|
|
|
|
the set. So even if you restore a single VM, this may have to
|
|
|
|
read data from all tapes inside the media set.
|
|
|
|
|
|
|
|
Larger media sets are also more error prone, because a single
|
|
|
|
damaged media makes the restore fail.
|
|
|
|
|
|
|
|
Usage scenario: Mostly used with tape libraries, and you manually
|
|
|
|
trigger new set creation by running a backup job with the
|
|
|
|
``--export`` option.
|
|
|
|
|
2021-01-11 06:11:17 +00:00
|
|
|
.. NOTE:: Retention period starts with the existence of a newer
|
|
|
|
media set.
|
|
|
|
|
2021-01-08 15:53:46 +00:00
|
|
|
- Always create a new media set.
|
|
|
|
|
|
|
|
With this setting each backup job creates a new media set. This
|
|
|
|
is less space efficient, because the last media from the last set
|
|
|
|
may not be fully written, leaving the remaining space unused.
|
|
|
|
|
|
|
|
The advantage is that this procudes media sets of minimal
|
|
|
|
size. Small set are easier to handle, you can move sets to an
|
|
|
|
off-site vault, and restore is much faster.
|
|
|
|
|
2021-01-11 06:11:17 +00:00
|
|
|
.. NOTE:: Retention period starts with the creation time of the
|
|
|
|
media set.
|
|
|
|
|
2021-01-08 15:53:46 +00:00
|
|
|
- Create a new set when the specified Calendar Event triggers.
|
|
|
|
|
|
|
|
.. _systemd.time manpage: https://manpages.debian.org/buster/systemd/systemd.time.7.en.html
|
|
|
|
|
|
|
|
This allows you to specify points in time by using systemd like
|
|
|
|
Calendar Event specifications (see `systemd.time manpage`_).
|
|
|
|
|
|
|
|
For example, the value ``weekly`` (or ``Mon *-*-* 00:00:00``)
|
|
|
|
will create a new set each week.
|
|
|
|
|
|
|
|
This balances between space efficency and media count.
|
2021-01-08 07:46:25 +00:00
|
|
|
|
2021-01-11 06:11:17 +00:00
|
|
|
.. NOTE:: Retention period starts when the calendar event
|
|
|
|
triggers.
|
|
|
|
|
2021-01-08 07:46:25 +00:00
|
|
|
Additionally, the following events may allocate a new media set:
|
|
|
|
|
2021-01-08 15:53:46 +00:00
|
|
|
- Required tape is offline (and you use a tape library).
|
2021-01-08 07:46:25 +00:00
|
|
|
|
|
|
|
- Current set contains damaged of retired tapes.
|
|
|
|
|
2021-01-20 10:24:58 +00:00
|
|
|
- Media pool encryption changed
|
|
|
|
|
2021-01-08 15:53:46 +00:00
|
|
|
- Database consistency errors, e.g. if the inventory does not
|
|
|
|
contain required media info, or contain conflicting infos
|
|
|
|
(outdated data).
|
2021-01-08 07:46:25 +00:00
|
|
|
|
|
|
|
.. topic:: Retention Policy
|
|
|
|
|
2021-01-11 06:11:17 +00:00
|
|
|
Defines how long we want to keep the data.
|
2021-01-08 07:46:25 +00:00
|
|
|
|
2021-01-08 16:34:58 +00:00
|
|
|
- Always overwrite media.
|
|
|
|
|
|
|
|
- Protect data for the duration specified.
|
|
|
|
|
|
|
|
We use systemd like time spans to specify durations, e.g. ``2
|
|
|
|
weeks`` (see `systemd.time manpage`_).
|
|
|
|
|
|
|
|
- Never overwrite data.
|
2021-01-08 07:46:25 +00:00
|
|
|
|
2021-01-23 14:19:28 +00:00
|
|
|
.. topic:: Hardware Encryption
|
|
|
|
|
|
|
|
LTO4 (or later) tape drives support hardware encryption. If you
|
|
|
|
configure the media pool to use encryption, all data written to the
|
|
|
|
tapes is encrypted using the configured key.
|
|
|
|
|
|
|
|
That way, unauthorized users cannot read data from the media,
|
|
|
|
e.g. if you loose a media while shipping to an offsite location.
|
|
|
|
|
|
|
|
.. Note:: If the backup client also encrypts data, data on tape
|
|
|
|
will be double encrypted.
|
|
|
|
|
|
|
|
The password protected key is stored on each media, so it is
|
|
|
|
possbible to `restore the key <restore_encryption_key_>`_ using the password. Please make sure
|
|
|
|
you remember the password in case you need to restore the key.
|
|
|
|
|
2021-01-12 15:37:23 +00:00
|
|
|
|
2021-01-08 07:46:25 +00:00
|
|
|
.. NOTE:: FIXME: Add note about global content namespace. (We do not store
|
|
|
|
the source datastore, so it is impossible to distinguish
|
|
|
|
store1:/vm/100 from store2:/vm/100. Please use different media
|
|
|
|
pools if the source is from a different name space)
|
|
|
|
|
|
|
|
|
2021-01-12 15:37:23 +00:00
|
|
|
The following command creates a new media pool::
|
|
|
|
|
|
|
|
// proxmox-tape pool create <name> --drive <string> [OPTIONS]
|
|
|
|
|
|
|
|
# proxmox-tape pool create daily --drive mydrive
|
|
|
|
|
2021-01-08 07:46:25 +00:00
|
|
|
|
2021-01-12 15:37:23 +00:00
|
|
|
Additional option can be set later using the update command::
|
2021-01-08 07:46:25 +00:00
|
|
|
|
2021-01-12 15:37:23 +00:00
|
|
|
# proxmox-tape pool update daily --allocation daily --retention 7days
|
|
|
|
|
|
|
|
|
|
|
|
To list all configured pools use::
|
|
|
|
|
|
|
|
# proxmox-tape pool list
|
|
|
|
┌───────┬──────────┬────────────┬───────────┬──────────┐
|
|
|
|
│ name │ drive │ allocation │ retention │ template │
|
|
|
|
╞═══════╪══════════╪════════════╪═══════════╪══════════╡
|
|
|
|
│ daily │ mydrive │ daily │ 7days │ │
|
|
|
|
└───────┴──────────┴────────────┴───────────┴──────────┘
|
2021-01-08 07:46:25 +00:00
|
|
|
|
2020-12-20 08:16:09 +00:00
|
|
|
|
|
|
|
Tape Jobs
|
|
|
|
~~~~~~~~~
|
|
|
|
|
|
|
|
|
|
|
|
Administration
|
|
|
|
--------------
|
|
|
|
|
2021-01-08 18:17:31 +00:00
|
|
|
Many sub-command of the ``proxmox-tape`` command line tools take a
|
|
|
|
parameter called ``--drive``, which specifies the tape drive you want
|
|
|
|
to work on. For convenience, you can set that in an environment
|
|
|
|
variable::
|
|
|
|
|
|
|
|
# export PROXMOX_TAPE_DRIVE=mydrive
|
|
|
|
|
|
|
|
You can then omit the ``--drive`` parameter from the command. If the
|
|
|
|
drive has an associated changer device, you may also omit the changer
|
|
|
|
parameter from commands that needs a changer device, for example::
|
|
|
|
|
|
|
|
# proxmox-tape changer status
|
|
|
|
|
|
|
|
Should displays the changer status of the changer device associated with
|
|
|
|
drive ``mydrive``.
|
|
|
|
|
|
|
|
|
2020-12-20 08:16:09 +00:00
|
|
|
Label Tapes
|
|
|
|
~~~~~~~~~~~
|
|
|
|
|
2020-12-20 09:41:40 +00:00
|
|
|
By default, tape cartidges all looks the same, so you need to put a
|
|
|
|
label on them for unique identification. So first, put a sticky paper
|
|
|
|
label with some human readable text on the cartridge.
|
|
|
|
|
|
|
|
If you use a `Tape Library`_, you should use an 8 character string
|
|
|
|
encoded as `Code 39`_, as definded in the `LTO Ultrium Cartridge Label
|
|
|
|
Specification`_. You can either bye such barcode labels from your
|
|
|
|
cartidge vendor, or print them yourself. You can use our `LTO Barcode
|
|
|
|
Generator`_ App for that.
|
|
|
|
|
|
|
|
Next, you need to write that same label text to the tape, so that the
|
|
|
|
software can uniquely identify the tape too.
|
|
|
|
|
|
|
|
For a standalone drive, manually insert the new tape cartidge into the
|
2021-01-08 18:17:31 +00:00
|
|
|
drive and run::
|
2020-12-20 09:41:40 +00:00
|
|
|
|
2021-01-12 15:37:23 +00:00
|
|
|
# proxmox-tape label --changer-id <label-text> [--pool <pool-name>]
|
|
|
|
|
|
|
|
You may omit the ``--pool`` argument to allow the tape to be used by any pool.
|
2020-12-20 09:41:40 +00:00
|
|
|
|
|
|
|
.. Note:: For safety reasons, this command fails if the tape contain
|
|
|
|
any data. If you want to overwrite it anways, erase the tape first.
|
|
|
|
|
2021-01-08 18:17:31 +00:00
|
|
|
You can verify success by reading back the label::
|
2020-12-20 09:41:40 +00:00
|
|
|
|
2021-01-12 15:37:23 +00:00
|
|
|
# proxmox-tape read-label
|
|
|
|
┌─────────────────┬──────────────────────────────────────┐
|
|
|
|
│ Name │ Value │
|
|
|
|
╞═════════════════╪══════════════════════════════════════╡
|
|
|
|
│ changer-id │ vtape1 │
|
|
|
|
├─────────────────┼──────────────────────────────────────┤
|
|
|
|
│ uuid │ 7f42c4dd-9626-4d89-9f2b-c7bc6da7d533 │
|
|
|
|
├─────────────────┼──────────────────────────────────────┤
|
|
|
|
│ ctime │ Wed Jan 6 09:07:51 2021 │
|
|
|
|
├─────────────────┼──────────────────────────────────────┤
|
|
|
|
│ pool │ daily │
|
|
|
|
├─────────────────┼──────────────────────────────────────┤
|
|
|
|
│ media-set-uuid │ 00000000-0000-0000-0000-000000000000 │
|
|
|
|
├─────────────────┼──────────────────────────────────────┤
|
|
|
|
│ media-set-ctime │ Wed Jan 6 09:07:51 2021 │
|
|
|
|
└─────────────────┴──────────────────────────────────────┘
|
|
|
|
|
|
|
|
.. NOTE:: The ``media-set-uuid`` using all zeros indicates an empty
|
|
|
|
tape (not used by any media set).
|
2020-12-20 09:41:40 +00:00
|
|
|
|
|
|
|
If you have a tape library, apply the sticky barcode label to the tape
|
|
|
|
cartridges first. Then load those empty tapes into the library. You
|
2021-01-08 18:17:31 +00:00
|
|
|
can then label all unlabeled tapes with a single command::
|
2020-12-20 09:41:40 +00:00
|
|
|
|
2021-01-12 15:37:23 +00:00
|
|
|
# proxmox-tape barcode-label [--pool <pool-name>]
|
2020-12-20 09:41:40 +00:00
|
|
|
|
|
|
|
|
2020-12-20 08:16:09 +00:00
|
|
|
Run Tape Backups
|
|
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
|
2021-01-12 16:26:15 +00:00
|
|
|
To manually run a backup job use::
|
|
|
|
|
|
|
|
# proxmox-tape backup <store> <pool> [OPTIONS]
|
|
|
|
|
|
|
|
The following options are available:
|
|
|
|
|
|
|
|
--eject-media Eject media upon job completion.
|
|
|
|
|
|
|
|
It is normally good practice to eject the tape after use. This unmounts the
|
|
|
|
tape from the drive and prevents the tape from getting dirty with dust.
|
|
|
|
|
|
|
|
--export-media-set Export media set upon job completion.
|
|
|
|
|
|
|
|
After a sucessful backup job, this moves all tapes from the used
|
|
|
|
media set into import-export slots. The operator can then pick up
|
|
|
|
those tapes and move them to a media vault.
|
|
|
|
|
|
|
|
|
2020-12-20 08:16:09 +00:00
|
|
|
Restore from Tape
|
|
|
|
~~~~~~~~~~~~~~~~~
|
|
|
|
|
2021-01-15 08:44:20 +00:00
|
|
|
Restore is done at media-set granularity, so you first need to find
|
|
|
|
out which media set contains the data you want to restore. This
|
|
|
|
information is stored in the media catalog. If you do not have media
|
|
|
|
catalogs, you need to restore them first. Please note that you need
|
|
|
|
the catalog to find your data, but restoring a complete media-set does
|
|
|
|
not need media catalogs.
|
|
|
|
|
|
|
|
The following command shows the media content (from catalog)::
|
|
|
|
|
|
|
|
# proxmox-tape media content
|
|
|
|
┌────────────┬──────┬──────────────────────────┬────────┬────────────────────────────────┬──────────────────────────────────────┐
|
|
|
|
│ label-text │ pool │ media-set-name │ seq-nr │ snapshot │ media-set-uuid │
|
|
|
|
╞════════════╪══════╪══════════════════════════╪════════╪════════════════════════════════╪══════════════════════════════════════╡
|
|
|
|
│ TEST01L8 │ p2 │ Wed Jan 13 13:55:55 2021 │ 0 │ vm/201/2021-01-11T10:43:48Z │ 9da37a55-aac7-4deb-91c6-482b3b675f30 │
|
|
|
|
├────────────┼──────┼──────────────────────────┼────────┼────────────────────────────────┼──────────────────────────────────────┤
|
|
|
|
│ ... │ ... │ ... │ ... │ ... │ ... │
|
|
|
|
└────────────┴──────┴──────────────────────────┴────────┴────────────────────────────────┴──────────────────────────────────────┘
|
|
|
|
|
|
|
|
|
|
|
|
A restore job reads the data from the media set and moves it back to
|
|
|
|
data disk (datastore)::
|
|
|
|
|
|
|
|
// proxmox-tape restore <media-set-uuid> <datastore>
|
|
|
|
|
|
|
|
# proxmox-tape restore 9da37a55-aac7-4deb-91c6-482b3b675f30 mystore
|
|
|
|
|
|
|
|
|
2020-12-20 08:16:09 +00:00
|
|
|
Update Inventory
|
|
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
|
2021-01-15 08:44:20 +00:00
|
|
|
|
2020-12-20 08:16:09 +00:00
|
|
|
Restore Catalog
|
|
|
|
~~~~~~~~~~~~~~~
|
2021-01-12 16:26:15 +00:00
|
|
|
|
2021-01-15 08:44:20 +00:00
|
|
|
|
2021-01-23 14:19:28 +00:00
|
|
|
Encryption Key Management
|
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
Creating a new encryption key::
|
|
|
|
|
|
|
|
# proxmox-tape key create --hint "tape pw 2020"
|
|
|
|
Tape Encryption Key Password: **********
|
|
|
|
Verify Password: **********
|
|
|
|
"14:f8:79:b9:f5:13:e5:dc:bf:b6:f9:88:48:51:81:dc:79:bf:a0:22:68:47:d1:73:35:2d:b6:20:e1:7f:f5:0f"
|
|
|
|
|
|
|
|
List existing encryption keys::
|
|
|
|
|
|
|
|
# proxmox-tape key list
|
|
|
|
┌───────────────────────────────────────────────────┬───────────────┐
|
|
|
|
│ fingerprint │ hint │
|
|
|
|
╞═══════════════════════════════════════════════════╪═══════════════╡
|
|
|
|
│ 14:f8:79:b9:f5:13:e5:dc: ... :b6:20:e1:7f:f5:0f │ tape pw 2020 │
|
|
|
|
└───────────────────────────────────────────────────┴───────────────┘
|
|
|
|
|
|
|
|
To show encryption key details::
|
|
|
|
|
|
|
|
# proxmox-tape key show 14:f8:79:b9:f5:13:e5:dc:...:b6:20:e1:7f:f5:0f
|
|
|
|
┌─────────────┬───────────────────────────────────────────────┐
|
|
|
|
│ Name │ Value │
|
|
|
|
╞═════════════╪═══════════════════════════════════════════════╡
|
|
|
|
│ kdf │ scrypt │
|
|
|
|
├─────────────┼───────────────────────────────────────────────┤
|
|
|
|
│ created │ Sat Jan 23 14:47:21 2021 │
|
|
|
|
├─────────────┼───────────────────────────────────────────────┤
|
|
|
|
│ modified │ Sat Jan 23 14:47:21 2021 │
|
|
|
|
├─────────────┼───────────────────────────────────────────────┤
|
|
|
|
│ fingerprint │ 14:f8:79:b9:f5:13:e5:dc:...:b6:20:e1:7f:f5:0f │
|
|
|
|
├─────────────┼───────────────────────────────────────────────┤
|
|
|
|
│ hint │ tape pw 2020 │
|
|
|
|
└─────────────┴───────────────────────────────────────────────┘
|
|
|
|
|
2021-01-23 14:34:28 +00:00
|
|
|
The ``paperkey`` subcommand can be used to create a QR encoded
|
|
|
|
version of a tape encryption key. The following command sends the output of the
|
|
|
|
``paperkey`` command to a text file, for easy printing::
|
|
|
|
|
|
|
|
proxmox-tape key paperkey <fingerprint> --output-format text > qrkey.txt
|
|
|
|
|
|
|
|
|
2021-01-23 14:19:28 +00:00
|
|
|
.. _restore_encryption_key:
|
|
|
|
|
|
|
|
Restoring Encryption Keys
|
|
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
|
|
|
|
You can restore the encryption key from the tape, using the password
|
|
|
|
used to generate the key. First, load the tape you want to restore
|
|
|
|
into the drive. Then run::
|
|
|
|
|
|
|
|
# proxmox-tape key restore
|
|
|
|
Tepe Encryption Key Password: ***********
|
|
|
|
|
|
|
|
If the password is correct, the key will get imported to the
|
2021-01-23 14:34:28 +00:00
|
|
|
database. Further restore jobs automatically use any availbale key.
|
2021-01-23 14:19:28 +00:00
|
|
|
|
2021-01-15 08:44:20 +00:00
|
|
|
|
2021-01-12 16:26:15 +00:00
|
|
|
Tape Cleaning
|
|
|
|
~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
LTO tape drives requires regular cleaning. This is done by loading a
|
|
|
|
cleaning cartridge into the drive, which is a manual task for
|
|
|
|
standalone drives.
|
|
|
|
|
|
|
|
For tape libraries, cleaning cartridges are identified using special
|
|
|
|
labels starting with letters "CLN". For example, our tape library has a
|
|
|
|
cleaning cartridge inside slot 3::
|
|
|
|
|
|
|
|
# proxmox-tape changer status sl3
|
|
|
|
┌───────────────┬──────────┬────────────┬─────────────┐
|
|
|
|
│ entry-kind │ entry-id │ changer-id │ loaded-slot │
|
|
|
|
╞═══════════════╪══════════╪════════════╪═════════════╡
|
|
|
|
│ drive │ 0 │ vtape1 │ 1 │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ slot │ 1 │ │ │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ slot │ 2 │ vtape2 │ │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ slot │ 3 │ CLN001CU │ │
|
|
|
|
├───────────────┼──────────┼────────────┼─────────────┤
|
|
|
|
│ ... │ ... │ │ │
|
|
|
|
└───────────────┴──────────┴────────────┴─────────────┘
|
|
|
|
|
|
|
|
To initiate a cleaning operation simply run::
|
|
|
|
|
|
|
|
# proxmox-tape clean
|
|
|
|
|
|
|
|
This command does the following:
|
|
|
|
|
|
|
|
- find the cleaning tape (in slot 3)
|
|
|
|
|
|
|
|
- unload the current media from the drive (back to slot1)
|
|
|
|
|
|
|
|
- load the cleaning tape into the drive
|
|
|
|
|
|
|
|
- run drive cleaning operation
|
|
|
|
|
|
|
|
- unload the cleaning tape (to slot 3)
|
2021-02-12 10:50:56 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Configuration Files
|
|
|
|
-------------------
|
|
|
|
|
|
|
|
``media-pool.cfg``
|
|
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
File Format
|
|
|
|
^^^^^^^^^^^
|
|
|
|
|
|
|
|
.. include:: config/media-pool/format.rst
|
|
|
|
|
|
|
|
|
|
|
|
Options
|
|
|
|
^^^^^^^
|
|
|
|
|
|
|
|
.. include:: config/media-pool/config.rst
|
2021-02-12 11:09:08 +00:00
|
|
|
|
|
|
|
|
|
|
|
``tape.cfg``
|
|
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
|
|
File Format
|
|
|
|
^^^^^^^^^^^
|
|
|
|
|
|
|
|
.. include:: config/tape/format.rst
|
|
|
|
|
|
|
|
|
|
|
|
Options
|
|
|
|
^^^^^^^
|
|
|
|
|
|
|
|
.. include:: config/tape/config.rst
|
2021-02-12 16:45:55 +00:00
|
|
|
|
|
|
|
|
|
|
|
Command Syntax
|
|
|
|
--------------
|
|
|
|
|
2021-02-12 17:00:01 +00:00
|
|
|
``proxmox-tape``
|
|
|
|
----------------
|
|
|
|
|
|
|
|
.. include:: proxmox-tape/synopsis.rst
|
|
|
|
|
|
|
|
|
2021-02-12 16:45:55 +00:00
|
|
|
``pmt``
|
|
|
|
--------
|
|
|
|
|
|
|
|
.. include:: pmt/options.rst
|
|
|
|
|
|
|
|
....
|
|
|
|
|
|
|
|
.. include:: pmt/synopsis.rst
|
|
|
|
|
|
|
|
|
|
|
|
``pmtx``
|
|
|
|
--------
|
|
|
|
|
|
|
|
.. include:: pmtx/synopsis.rst
|