CSIM Technical Notes

The system chg-vdisk

Amanda introduced virtual tapes (vtape) to store backups on a disk instead of magnetc tapes. At the time, Amanda could pnly use a single disk to store virtual tapes; if you had several disks, it was suggested to use some disk concatenation like RAID 0.

While a solution, I think that it is not the best solution: if a single disk is to fail, the RAID is not available anymore and one lose all the backups. Using a RAID of higher level is not ideal, I would prefer to trade the reliability of RAID for more backup storage and more incremental backups.

To that effect, I have modified the tape changer for vtapes chg-disk to work on a bunch of disks.

Each disk is mounted at a mounting point in Unix. At this mounting point there must be a directory amanda that belongs to the user and group running Amanda server.

In the directory amanda there must exist a file called disklabel that contains a unique name for that disk that will be used further in the configuration. The directory amanda also contains the slots as defined by the vtapes. The slots are numbered in sequence across all the vdisks, 1 to a for the first disk, a+1 to b for the second disk, b+1 to c for the third, etc.

A configuration chg-vdisk.conf file must be created in Amanda configuration directory (where the file amanda.conf is saved). It will be described later.

Two files, chg-vdisk.cache.db and chg-vdisk.current, are created by chg-vdisk in that same Amanda configuration directory and should not be changed by the user.

The configuration file chg-vdisk.conf

The configuration file of chg-vdisk resides in the Amanda configuration directory and follow the syntax of Config::IniFiles of Perl, with sections between square brackets ([...]) and a list of parameter=value.

• [syslog] defines options used by syslog:
  • facility= the facility used to log syslog messages.
• [options] defines general options:
  • path= the path of Amanda configuration directory;
  • pid_file= the path of the file to store chg-vdisk pid;
  • lock_file= the path of the file that chg-vdisk can use for locking operation;
  • timeout= (obsolete, see below) the timeout in seconds to wait for autmount;
  • signal= (obsolete) the signal thatthe operator can send to chg-vdisk once the requested disk has been mounted.
• [mountpoints] defines the list of mount points used by the disks, each parameter if of the form:
  • /path/to/the/mountpoint=1 where /path/to/the/mountpoint is the equivalent of the disk mountpoint in /etc/fstab. Note that the =1 is dictated by the file format that must associate a value to each entry.
• [disklabel] for each disk in the system, there is a section named by the disklabel of the disk:
  • first-slot= is the first slot that exist on that disk;
  • last-slot= is the last slot that exist on that disk.
• [email] (obsolete) defines the email address of the operator:
  • mailto= is the email address used for the recipient, uses Amanda configuration value by default;
  • mailfrom= is the email address used for the sender;
  • mailrepeat= is the frequency, in seconds, to send repeated email.
• [week] and [holiday] (obsolete) define the date and time when an operator is present and can swap disks (see below). The syntax is equivalent to the one used for CSIM access control.

Adding a disk to the system

This presuppose that the disk is running and attached to the Unix server.

If you want to add a new disk to the system, there are two steps for initialisation of the new vdisk and one step for mounting the disk; if you need to mount a disk that has already been initialised and used, for example an old disk that has been retired but still holds backups, jump to the step 3:

1. initialize the new vdisk and create the slots:

   You must create directory amanda and the file disklabel at the root of the vdisk; these must belong to the user and group running Amanda.

   Then the slots are created with the script ~on/create_slot:

   sudo ~on/create_slot disklabel #slot

   where disklabel is the label used in the file disklabel and #slot is the number of slots you want to create on that disk.

   For CSIM, each vtape is 110GB, so a 6TB disk would allow about 53 slots.

   Note that the slots are created with number starting at zero;

2. mount the vdisk:

   This step renumbers all the slots of the vdisks accessible on the server to make sure that the slot number are in sequence, this is because Amanda tape changer expects slots that are numbered in sequence from 1 to the maximum while adding a disk may introduce gaps or duplicated.

   This script does not change the vtapes or the contents of the backups that are present on the disks, only the slot numbers are modified, it is safe to run this scipt on disks that alread contain backups.

   The script ~on/renum_slot do the renumbering and issue a piece of configuration that must be updated manually inside chg-vdisk.conf. This script nust be run by root.

   You can simply issue the command:

   sudo ~on/renum_slot

   as a result you will get some information of the form:

   [disk CSIM-set-10]
   # mounted on /host/localhost/ada2
   first-slot=1
   last-slot=27
   
   [disk CSIM-set-11]
   # mounted on /host/localhost/ada1
   first-slot=28
   last-slot=54
   
   [disk CSIM-set-13]
   # mounted on /host/localhost/ada7
   first-slot=55
   last-slot=72
   

   This is to replace the corresponding section in chg-vdisk configuration file;

3. initialize new new vtapes in the slots of the new vdisk:

   The script ~on/label_slot creates new vtapes in each slots of the disk disklabel; it uses amlabel internally and must be run by the user amanda.

   You can run the command:

   sudo -u amanda ~on/label_slot disklabel

   where disklabel is the same as above. Vtapes will have a sequential number starting from the maximum number found in the file tapelist.

Removing a disk from the system

It may be necessary to remove a disk from the system if the disk has failed and became unusable or if a newer and larger disk would be installed.

Once the disk jas been physically removed from the server, two steps must be done:

1. renumber the slots for the remaining disks:

   The script ~on/renum_slot is used and the output must be imported into chg-vdisk configuration, as described above;

2. make Amanda aware that the vtapes are not mounted in any disk slot anymore and should be marked no-reuse:

   The script ~on/unlabel_slot can be used to mark the vtapes with no-reuse; it lists the vtapes marked reuse in the file tapelist and checks that the vtape is present in a slot.

   If not, the vtape is marked no-reuse (by using the command amadmin).

   This script must be run by root.

Disk automouting on FreeBSD

The other files created by chg-vdisk

• chg-vdisk.current: the notion of a tape currently loaded in the tape changer is moot for vtapes, but it is part of the API for the tape changers in Amanda. This file keeps track of the tape "currently" in the changer.
• chg-vdisk.cache.db: the system needs to keep a link between the slots and the vtape present in each slot. This information does not change frequently so it would a waste of resources to compute it again each time the the vdisk changer is called. This database keeps track of the vtapes mounted in the slots.

  When a certain vtape is needed, the slot number in the chache is checked first, and if the vtape on disk at this slot is the expected one, chg-vdisk continues its its operation. Else, if the vtape in the slot is different from the one cached, all the disks are read, one slot at a time and a new cache is saved.

Some other features

i originally envisioned chg-vdisk to work with USB attached disks, where a limited number of USB slots were available. The system would have been able to find out which of the disks were currently pluged in and request an operator to switch disk appropriately.

USB disk were slow and SATA interface rendered all that part very obsolete. But that is the reason of the part regarding email and holidays in the configuration file.

Finally, chg-vdisk should be modified to run as a multi-changer as it can write on several disks or files at the same time, but that has never been the bottleneck with vtapes, so it runs one vtape at a time.