Amanda operator

This page provides more information for Amanda operators.

It is intended to be a reference manual for Amanda.

Many changes and additions to this page; Amanda index database; erasing a tape; system reboot; data recovery; building Amanda client New...

CSIM Logo WelcomeCourses
Faculty, Student, Staff
Projects and reports
Conferences, workshop and seminars
Laboratories and reasearch facilities
Information related to CSIM
Information non-related to CSIM
Address, map, phone, etc.
Search

Back-up schedule

Back-up are run 5 nights a week, on Tuesday to Saturday, and start at 0:45. This schedule allows to load a tape during the previous working day.

Back-up reminders

The command:

sudo -u amanda amrotate -r normal

(where normal is the name of Amanda configuration) is used to check that the correct tape cartridge is loaded in the tape drive and to send a reminder to Amanda operators.

Reminders are sent on Monday to Friday, starting at 11:00, to inform the operators about the tape to load. Email received will looks like:

Date: Wed, 6 Jun 2001 11:01:56 +0700 (ICT)
From: Amanda user <amanda@cs.ait.ac.th>
To: adul@cs.ait.ac.th, dipak@cs.ait.ac.th, on@cs.ait.ac.th
Subject: Amanda check tape: MOUNT CSIM-set-1-03

Please mount the tape CSIM-set-1-03.

The line Mount the tape CSIM-set-1-03 shows the name of the tape that should be inserted. Note that on the tape itself, the label will say 1-03 or 103.

If the option -r is used, email will be repeated every hour, then every half hour until the correct tape is loaded into the tape drive. No reminder email is sent if the correct tape is already found in the drive.

If the tape drive is busy, the email will only inform of the tape that is expected, and messages will be repeated until the tape can be ckecked.

If the tape could be checked and it is not the expected tape, it will be ejected from the drive.

Back-up of servers

Amanda is planning the full and incremental back-up to share the load along one week. Disks holding system data are backed-up only once a week. Disks holding users data, including email, database, etc. are backed-up every day.

Back-up of PC's

PC's under windows (95, 98, NT, 2000) can be back-up too. Up to 2 shares can be saved daily and up to 3 shares can be saved weekly. The limit on the number of shares may change depending on the load on Amanda. (A share is a windows directory that is shared for others to use.)

Users can decide what they want to back-up on their PC by accessing the configuration page. They are responsible to stick to the rules regarding the name of shares, users and passwords.

In Amanda logs, the Windows shares will be called like:

amanda     //multimedia/temp

Where it represents the share temp on the Windows 98 machine multimedia.

Handling tapes

Inserting tapes

The tape should be piled up in order on the side of the Amanda server. Remove the tape from the drive, put it in a box and place it at the bottom of the pile.

In case the tape is not automatically ejected from the drive, push on the eject button, in red, located on the right of the drive.

Take the tape from the top and insert it in the drive. When inserting the tape in the drive, make sure that the label is on the top. On the top of the cartridge, 3 elements will show the right direction:

Push the tape gently in the drive, the drive will automatically insert the tape. Make sure you push the cartridge straight, not to the right or to the left, else it will not be taken by the mechanism.

Never open the shutter, it protects the tape from dust.

Drive cleaning

Every time a new tape set is used, the drive must be cleaned. Cleaning should occur before tape 1-00, 2-00 and 3-00. Cleaning tape is recognizable by the different label.

The drive must be cleaned every week to ensure a reliable back-up. Just insert the cleaning cartridge in the drive. After 20 seconds it will be automatically ejected. Record the date on the cleaning cartridge label, and proceed to insert the proper tape for the next back-up.

The orange LED on the drive indicates that cleaning must be performed. In any case, preventive cleaning should be conducted every week.

Reading logs

While Amanda is running

It is possible to check the status of a back-up, while Amanda is still running:

bazooka<on>45: amstatus normal
Using /amanda/CSIM-set-1/amdump from Tue Jun 19 00:45:01 ICT 2001

amanda:/                                 0   11712k finished (1:25:07)
amanda://amy/mydocuments                 0 no estimate
ufo:/usr                                 0  274435k wait for dumping 
ufo:/web                                 0  651190k dumping   538144k ( 82.64%) (8:56:36)

This first section indicates that the disk / from machine amanda was backep-up properly. The file system //amy/mydocuments could not be checked, it will produce an error in Amanda report later on. For the machine ufo, one disk is being dumped while another one is waiting.

SUMMARY          part     real estimated
                          size      size
partition       :   4
estimated       :   3           11885333k
failed          :   1                  0k           (  0.00%)
wait for dumping:   1                  0k           (  0.00%)
dumping to tape :   0                  0k           (  0.00%)
dumping         :   1   538144k   651190k ( 82.64%) (  4.53%)
dumped          :   1 11209280k 11234143k ( 99.78%) ( 94.31%)
wait for writing:   0        0k        0k (  0.00%) (  0.00%)
writing to tape :   0        0k        0k (  0.00%) (  0.00%)
failed to tape  :   0        0k        0k (  0.00%) (  0.00%)
taped           :   1 11209280k 11234143k ( 99.78%) ( 94.31%)

This section summarize what have been dumped and what is left waiting. One disk has been dumped and written to tape, one disk is dumping, and one is waiting. It also summarize the data size.

11 dumpers idle  : not-idle
taper idle
network free kps:   496344
holding space   : 11891273k ( 94.81%)
 dumper0 busy   :  2:22:55  ( 31.48%)
 dumper1 busy   :  1:47:15  ( 23.62%)
 dumper2 busy   :  1:12:06  ( 15.88%)
 dumper3 busy   :  7:01:36  ( 92.86%)
 dumper4 busy   :  0:44:09  (  9.73%)
 dumper5 busy   :  0:27:21  (  6.02%)
   taper busy   :  0:42:51  (  9.44%)
 0 dumpers busy :  0:00:00  (  0.00%)
 1 dumper busy  :  5:39:17  ( 74.73%)  client-constrained:  5:37:31  ( 99.48%)
                                                 not-idle:  0:01:00  (  0.29%)
                                               start-wait:  0:00:46  (  0.23%)
 2 dumpers busy :  0:07:15  (  1.60%)  client-constrained:  0:07:15  (100.00%)
 3 dumpers busy :  0:38:47  (  8.55%)  client-constrained:  0:38:40  ( 99.68%)
                                               start-wait:  0:00:07  (  0.32%)
 4 dumpers busy :  0:24:39  (  5.43%)  client-constrained:  0:24:36  ( 99.80%)
                                               start-wait:  0:00:02  (  0.20%)
 5 dumpers busy :  0:17:19  (  3.82%)  client-constrained:  0:17:16  ( 99.67%)
                                               start-wait:  0:00:03  (  0.33%)
 6 dumpers busy :  0:26:40  (  5.88%)  client-constrained:  0:24:28  ( 91.74%)
                                             no-bandwidth:  0:02:10  (  8.14%)
                                               start-wait:  0:00:01  (  0.12%)
bazooka<on>46: 

This last section indicates the use of parallel dumpers, as well as the usage of the holding disk to store data temporarilly on the back-up server.

In case Amanda is not doing a back-up, the command will give the following error:

bazooka<on>49: amstatus normal   
/amanda/CSIM-set-1/amdump: No such file or directory at /usr/local/sbin/amstatus line 103.
bazooka<on>50:  

Report at the end of Amanda dump

Amanda logs look like the following example:

Date: Tue, 5 Jun 2001 15:19:03 +0700 (ICT)
From: Amanda user <amanda@cs.ait.ac.th>
To: adul@cs.ait.ac.th, dipak@cs.ait.ac.th, on@cs.ait.ac.th
Subject: CSIM AMANDA MAIL REPORT FOR June 5, 2001

These dumps were to tape CSIM-set-1-02.
The next tape Amanda expects to use is: CSIM-set-1-03.

FAILURE AND STRANGE DUMP SUMMARY:
  amanda     //dolphin/mssql7 lev 0 STRANGE

STATISTICS:
                          Total       Full      Daily
                        --------   --------   --------
Estimate Time (hrs:min)    0:27
Run Time (hrs:min)         1:40
Dump Time (hrs:min)        1:09       0:59       0:11
Output Size (meg)        1510.2     1381.3      128.9
Original Size (meg)      4150.7     3923.7      227.0
Avg Compressed Size (%)    36.4       35.2       56.8   (level:#disks ...)
Filesystems Dumped           11          3          8   (1:7 2:1)
Avg Dump Rate (k/s)       372.1      401.9      207.6

Tape Time (hrs:min)        0:05       0:05       0:01
Tape Size (meg)          1510.6     1381.4      129.2
Tape Used (%)               3.3        3.0        0.3   (level:#disks ...)
Filesystems Taped            11          3          8   (1:7 2:1)
Avg Tp Write Rate (k/s)  4767.5     5030.3     3058.5

FAILED AND STRANGE DUMP DETAILS:

/-- amanda     //dolphin/mssql7 lev 0 STRANGE
sendbackup: start [amanda://dolphin/mssql7 level 0]
sendbackup: info BACKUP=/usr/local/bin/smbclient
sendbackup: info RECOVER_CMD=/usr/local/bin/smbclient -f... -
sendbackup: info end
| added interface ip=192.41.170.11 bcast=192.41.170.255 nmask=255.255.255.0
? ERRDOS - ERRnoaccess opening remote file \Binn\DbDgm1.sql (\Binn\)
? ERRDOS - ERRnoaccess opening remote file \Data\Inventory_Data.MDF (\Data\)
? ERRDOS - ERRnoaccess opening remote file \Data\Inventory_Log.LDF (\Data\)
| tar: dumped 2 files and directories
| Total bytes written: 0
sendbackup: size 0
sendbackup: end
\--------

NOTES:
  planner: Adding new disk amanda://dolphin/mssql7.
  planner: Adding new disk amanda://dolphin/temp.
  planner: Full dump of oak:/home/corse promoted from 2 days ahead.
  taper: tape CSIM-set-1-02 kb 1546848 fm 11 [OK]

DUMP SUMMARY:
                                     DUMPER STATS            TAPER STATS 
HOSTNAME     DISK        L ORIG-KB OUT-KB COMP% MMM:SS  KB/s MMM:SS  KB/s
-------------------------- --------------------------------- ------------
amanda       /           0 SKIPPED --------------------------------------
amanda       //csimnt1/c 0 SKIPPED --------------------------------------
amanda       //csimnt1/d 0 SKIPPED --------------------------------------
amanda       -hin/mssql7 0       0     32   --    0:00 110.6   0:02  35.9
amanda       -lphin/temp 0    3011   2336  77.6   0:03 865.7   0:011767.8
amanda       -ltimedia/d 1     190    192 101.1   0:13  14.3   0:01 283.3
.
.
.
(brought to you by Amanda version 2.4.2p2)

On a first glance, important elements are:

Tape verify report

After a backup has been written on a tape, the command:

sudo -u amanda amverify normal

(where normal is the name of Amanda configuration) will check that the data saved on the tape can be read properly.

Note: There is no way to check that the data are similar to the files on dick, because files on disk could have change in the mean time, but at least data on the tape are readable.

This command is run automatically after Amanda has finished backuping data every night.

Then Amanda will send the log of the tape verifying:

Date: Tue, 5 Jun 2001 15:43:45 +0700 (ICT)
From: Amanda user <amanda@cs.ait.ac.th>
To: adul@cs.ait.ac.th, dipak@cs.ait.ac.th, on@cs.ait.ac.th
Subject: CSIM AMANDA VERIFY REPORT FOR CSIM-set-1-02

Tapes:  CSIM-set-1-02
No errors found!

amverify normal
Tue Jun  5 15:34:50 ICT 2001

Using device /dev/nsa0
Volume CSIM-set-1-02, Date 20010605
Checked banyan._mysql.20010605.1
Checked amanda.__multimedia_temp.20010605.1
Checked amanda.__multimedia_mydocuments.20010605.1
Checked amanda.__dolphin_mssql7.20010605.0
Checked mail._var_mail.20010605.1
Checked amanda.__dolphin_temp.20010605.0
Checked oak._home_hawai.20010605.1
Checked amanda.__multimedia_d.20010605.1
Checked oak._home_java.20010605.1
Checked oak._home_fidji.20010605.2
Checked oak._home_corse.20010605.0
End-of-Tape detected.

It could contains indication about file number that is incorrect and have been reset. This is not an error.

Tapes:  CSIM-set-1-05
Errors found: 
CSIM-set-1-05 (amanda.__csimnt1_c.20010619.0):
amrestore: WARNING: not at start of tape, file numbers will be offset
amrestore:   0: restoring amanda.__csimnt1_c.20010619.0

gzip: stdin: decompression OK, trailing garbage ignored
/usr/bin/tar: Skipping to next header
/usr/bin/tar: Error exit delayed from previous errors
64+0 records in
64+0 records out
32768 bytes transferred in 0.011484 secs (2853384 bytes/sec)

amverify normal
Tue Jun 19 09:19:42 ICT 2001

Using device /dev/nsa0
Volume CSIM-set-1-05, Date 20010619
Checked mail._var_mail.20010619.1
Checked oak._usr.20010619.0
Checked mail._.20010619.0
Checked mail._var.20010619.0
** Error detected (amanda.__csimnt1_c.20010619.0)
amrestore: WARNING: not at start of tape, file numbers will be offset
amrestore:   0: restoring amanda.__csimnt1_c.20010619.0

gzip: stdin: decompression OK, trailing garbage ignored
/usr/bin/tar: Skipping to next header
/usr/bin/tar: Error exit delayed from previous errors
64+0 records in
64+0 records out
32768 bytes transferred in 0.011484 secs (2853384 bytes/sec)
Checked dns._usr.20010619.0
Checked amanda._usr.20010619.0

End-of-Tape detected.

This warning message should not influence on the reliability of the baked-up data.

Database report

The index database holds information about what tape has what backup, for what disk, on what date and on what incremental level. Amanda configuration files may be of some importance too, especially the tapelist file that should indicate what tape Amanda will use in the next backup. It is important that this information could be recovered from a secondary source in case Amanda server get corrupted. In CSIM, this information is saved into two different ways:

  1. the database in /amanda and the configuration in /usr/local/etc/amanda are rsync'ed to database:/database/amanda;
  2. the result of amadmin export is emailed to Amanda operators; the import command could be used to re-create thedatabase; the contents of the tapelist file is emailed to Amanda operators.

The command:

sudo -u amanda amdatabase normal

(where normal is the name of amanda configuration) is in charge of saving the index database.

Amanda operators receive an email like:

Date: Fri, 24 Mar 2006 10:41:11 +0700 (ICT)
From: Amanda user <amanda@cs.ait.ac.th>
To: on@cs.ait.ac.th, dipak@cs.ait.ac.th, adul@cs.ait.ac.th
Subject: AMANDA DATABASE INFO

Tapelist after the last write: /usr/local/etc/amanda/tapelist
------
20060427 CSIM-set-1-04 reuse
20060426 CSIM-set-1-03 reuse
20060423 CSIM-set-1-02 reuse ...
------
RSynchronize the amanda:/usr/local/etc/amanda to database:/database/amanda
RSynchronize the amanda:/var/amanda to database:/database/amanda
------
building file list ... done CSIM-set-1/ CSIM-set-1/amflush.1 CSIM-set-1/amflush.2 CSIM-set-1/amflush.3 ... CSIM-set-1/curinfo/ufo1000/_usr/info sent 69733 bytes received 814 bytes 1205.93 bytes/sec total size is 207378651 speedup is 2939.58 ------ Copy of Amanda info database Save the file and use "amadmin import" when you need to recreate the database on Amanda server ------ This part has been generated with amadmin export CURINFO Version 2.4.2p2 CONF CSIM # Generated by: # host: amanda.cs.ait.ac.th # date: Fri Mar 24 10:41:11 2006 # command: /usr/local/sbin/amadmin normal export # This file can be merged back in with "amadmin import". # Edit only with care. host: amanda disk: / command: 0 last_level: 0 ...

This email will be needed only is Amanda server itself needs to be restored.

Trouble shooting

Error message
Description

A file has disappeared from the directory while GNU tar was performing the back-up:

FAILURE AND STRANGE DUMP SUMMARY:
  ufo        /news lev 0 STRANGE


/-- ufo        /news lev 0 STRANGE
sendbackup: start [ufo:/news level 0]
sendbackup: info BACKUP=/usr/local/bin/tar
sendbackup: info RECOVER_CMD=/usr/local/bin/tar -f... -
sendbackup: info end
? gtar: ./comp/security/17979: Warning: Cannot stat: 
        No such file or directory
| Total bytes written: 3894374400 (3.6GB, 150kB/s)
sendbackup: size 3803100
sendbackup: end
\--------

On the machine ufo, the file /news/comp/security/17979 has disappeared when GNU tar was trying to back-up it.

The file was listed in the directory, but the file itself is not present.

GNU tar will skip this file.

In this case, a News article did expire during back-up time, it is expected to be a common occurance.

The contents of a file has changed while GNU tar was performing the back-up:

FAILURE AND STRANGE DUMP SUMMARY:
  mail /var/mail          lev 0 STRANGE 
FAILED AND STRANGE DUMP DETAILS:
/-- mail       /var/mail lev 0 STRANGE
sendbackup: start [mail:/var/mail level 0]
sendbackup: info BACKUP=/usr/local/bin/tar
sendbackup: info RECOVER_CMD=/usr/local/bin/tar -f... -
sendbackup: info end
? gtar: ./a01600: file changed as we read it
| Total bytes written: 377610240 (360MB, 412kB/s)
sendbackup: size 368760
sendbackup: end
\--------

On the machine mail, the file /bar/mail/a01600 has changed while it was getting saved.

GNU tar is able to recover from this error.

A Windows file is not readable:

FAILURE AND STRANGE DUMP SUMMARY:
  amanda     //dolphin/mssql7 lev 0 STRANGE

FAILED AND STRANGE DUMP DETAILS:
/-- amanda     //dolphin/mssql7 lev 0 STRANGE
sendbackup: start [amanda://dolphin/mssql7 level 0]
sendbackup: info BACKUP=/usr/local/bin/smbclient
sendbackup: info RECOVER_CMD=/usr/local/bin/smbclient -f
sendbackup: info end
| added interface ip=192.41.170.11 bcast=192.41.170.255 
|                                  nmask=255.255.255.0
? ERRnoaccess opening remote file \Binn\DbDgm1.sql
| tar: dumped 2 files and directories
| Total bytes written: 0
sendbackup: size 0
sendbackup: end
\--------

Amanda cannot back-up files that are already open by Windows.

On machine dolphin, the file \Binn\DbDgm1.sql could not be saved in the share mssql7.

This error is not recovered automatically.

A disk have been added while a back-up is running:

FAILURE AND STRANGE DUMP SUMMARY:
  cs4        /var             RESULTS MISSING

This is a warning message. The disk /var on the machine cs4 had been added while a back-up is running. Amanda is confused.

It will be corrected in the next run.

A client is not accessible:

FAILURE AND STRANGE DUMP SUMMARY:
  banyan     /mysql lev 0 FAILED [Request to banyan 
                                  timed out.]

The machine banyan could not be accessed. Either the machine is down, or the network is broken, or there is network permission conflict.

It will be corrected in the next run if the problem is solved.

Cannot access to a Windows share:

FAILURE AND STRANGE DUMP SUMMARY:
  amanda     //csimnt1/d lev 0 FAILED [disk //csimnt1/d 
                                       offline on amanda?]

The Windows share d on machine csimnt1 cannot be accessed. Either the machine is offline or the password is invalid.

It will recover on next run if the problem is solved.

No tape loaded in the drive:

*** A TAPE ERROR OCCURRED: [no tape online].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: CSIM-set-1-02.

There is no tape in the tape drive.

Mount the requested tape (in that case CSIM-set-1-02) and flush the holding disk.

Wrong tape in the drive:

*** A TAPE ERROR OCCURRED: 
            [cannot overwrite active tape CSIM-set-1-01].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: CSIM-set-1-03.

The tape in the drive is CSIM-set-1-01 and not the expected CSIM-set-1-03.

Mount the requested tape (in that case CSIM-set-1-03) and flush the holding disk.

Tape writting problem:

These dumps were to tape CSIM-set-1-04.
*** A TAPE ERROR OCCURRED: 
                [[writing filemark: Input/output error]].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: CSIM-set-1-05

FAILURE AND STRANGE DUMP SUMMARY:
  mail1000   /var lev 0 STRANGE
  sysl       /var/log lev 0 FAILED [out of tape]
  amanda     //olivier/drivec lev 0 STRANGE
...
STATISTICS:
                          Total       Full      Daily
                        --------   --------   --------
Estimate Time (hrs:min)    0:11
...
Tape Time (hrs:min)        0:03       0:02       0:02
Tape Size (meg) 574.1 308.8 265.3

There was an error while writting the dump sysl:/var/log of on the tape.

This is confirmed by the fact that only 574 MB were written to the tape, while a tape should hold 35 to 40 GB.

The tape is faulty, put a PostIt on the tape cartridge to reccord the problem; the PostIt should read "Tape ends after 574 MB. Check again on next run. Date". Ifthe problem repeats for that tape, the tape should be erased.

Mount the requested tape (in that case CSIM-set-1-05) and flush the holding disk.

amverify will display an error for the specific disk entry that caused the tape error.

Tape writting problem:

These dumps were to tape CSIM-set-1-04.
*** A TAPE ERROR OCCURRED: [[writing file: short write]].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: CSIM-set-1-05

FAILURE AND STRANGE DUMP SUMMARY:
  mail1000   /var lev 0 STRANGE
  sysl       /var/log lev 0 FAILED [out of tape]
  amanda     //olivier/drivec lev 0 STRANGE
...
STATISTICS:
                          Total       Full      Daily
                        --------   --------   --------
Estimate Time (hrs:min)    0:11
...
Tape Time (hrs:min)        0:03       0:02       0:02
Tape Size (meg) 574.1 308.8 265.3

There was an error while writting the dump sysl:/var/log of on the tape.

This could denote either that the tape is faulty or that Amanda reached the end of the tape.

Only 574 MB were written to the tape, while a tape should hold 35 to 40 GB, so there is a problem on the tape: put a PostIt on the tape cartridge to reccord the problem; the PostIt should read "Tape ends after 574 MB. Check again on next run. Date". Ifthe problem repeats for that tape, the tape should be erased.

In any case (tape faulty or not) mount the requested tape (in that case CSIM-set-1-05) and flush the holding disk.

amverify will display an error for the specific disk entry that caused the tape error.

Dump problem:

*** THE DUMPS DID NOT FINISH PROPERLY!

These dumps were to tape CSIM-set-1-02.
*** A TAPE ERROR OCCURRED: [[writing file: short write]].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next tape Amanda expects to use is: CSIM-set-1-03.

FAILURE AND STRANGE DUMP SUMMARY:
  amanda     /var lev 0 STRANGE
  amanda     /var lev 0 FAILED [out of tape]
  mail1000   /var RESULTS MISSING

Amanda server rebooted while it was collecting the backup information from the various clients.

What has been written on the tape is correct, but what is left on the holding disk is partial data and would not be of any use.

Remove any partial data from the holding disk in /holding/20060327 (if the problem happens on March 27th, 2006) and do not flush the holding disk.

Verify what could be saved on the tape (last file being broken most probably) and save the index database.

No writable tape while using vtape.
*** A TAPE ERROR OCCURRED: [No writable valid tape found].
Some dumps may have been left in the holding disk.
Run amflush to flush them to tape.
The next 5 tapes Amanda expects to use are: CSIM-set-3-20...
There seems to be an issue with the automound daemon that would mount USB disks on demand. For one of the disks, ls /dev/da* would return too many entries.
amanda<on>41: ls /dev/da*
/dev/da0     /dev/da0s1d 
/dev/da1s1c  /dev/da2s1
/dev/da0s1   /dev/da1 
/dev/da1s1d  /dev/da2s1c
/dev/da0s1c  /dev/da1s1 
/dev/da2     /dev/da2s1d

In case of error there are about 20 enties for /dev/da2. I rebooted the machine and flushed the holding disk.

Flush the holding disks

Sometime Amanda could not write all the data on a tape, either because the tape in the drive was not the expected tape, or because the tape ran full before all the data could be saved. In that case, dumps are kepts on the holding disk, in a directory called /holding/20060327 (for a run on March 27th,2006). To push these dumps onto a tape, you use the command amflush:

sudo -u amanda amflush normal

An email is sent to Amanda operators when all the dumps has been written to the tape.

Then you must verify that the tape is properly written and save the database of Amanda index:

sudo -u amanda amverify normal
sudo -u amanda amdatabase normal
sudo -u amanda amrotate normal

The last command is in charge of changing the set of tape when needed and sending an email to warn the operator of the next tape to load in the drive.

Erasing a tape

Sometime Amanda reports a tape to be faulty, it finds the end of the tape before the tape uses 40GB, or the label could not be read. It is time to erase the tape.

In the following example, I use the tape CSIM-set-2-05. When you are just about to use that tape for a new back-up, when email reminder asks for the tape CSIM-set-2-05, erase the tape with the command:

sudo -u amanda mt erase

Then use the newly erased tape in the procedure of replacing a tape describe below.

Replacing a tape

Sometime Amanda reports a tape to be faulty, the label could not be read on the tape, or the data could not be saved on the tape. It is time to replace the tape.

In order to keep the set of tapes consistant, the new tape should have the same label as the old tape. In the following example, I will use the label CSIM-set-2-05 as an example.

  1. Connect on Amanda server, first as yourself and use sudo -u amanda, with each command to run the command with Amanda priviledges.
  2. Edit the file /usr/local/etc/amanda/tapelist and remove the line containing the label. Usually the line is the last one of the file.
     
    20021115 CSIM-set-2-04 reuse                
    20021114 CSIM-set-2-03 reuse
    20021113 CSIM-set-2-02 reuse
    20021112 CSIM-set-2-01 reuse
    20021109 CSIM-set-2-00 reuse
    20021108 CSIM-set-1-05 no-reuse
    20021107 CSIM-set-1-04 no-reuse
    20021106 CSIM-set-1-03 no-reuse
    20021105 CSIM-set-1-02 no-reuse
    20021102 CSIM-set-1-01 no-reuse
    20021101 CSIM-set-1-00 no-reuse
    20021031 CSIM-set-3-05 no-reuse
    20021030 CSIM-set-3-04 no-reuse
    20021030 CSIM-set-3-03 no-reuse
    20021028 CSIM-set-3-02 no-reuse
    20021023 CSIM-set-3-01 no-reuse
    20021022 CSIM-set-3-00 no-reuse
    20021019 CSIM-set-2-05 reuse
    
     
  3. Edit the file /usr/local/etc/amanda/tapelist.yesterday in the same way. Usually the line is the one before the last.
  4. Prepare a new tape and write the label on it with the command: amlabel normal CSIM-set-2-05.
  5. Edit the file /usr/local/etc/amanda/tapelist again, note that the line with the label is back, but it is now at the top of the file, and the number at the begining is 0. Just move that line to the bottom of the file.
     
    20021115 CSIM-set-2-04 reuse                
    20021114 CSIM-set-2-03 reuse
    20021113 CSIM-set-2-02 reuse
    20021112 CSIM-set-2-01 reuse
    20021109 CSIM-set-2-00 reuse
    20021108 CSIM-set-1-05 no-reuse
    20021107 CSIM-set-1-04 no-reuse
    20021106 CSIM-set-1-03 no-reuse
    20021105 CSIM-set-1-02 no-reuse
    20021102 CSIM-set-1-01 no-reuse
    20021101 CSIM-set-1-00 no-reuse
    20021031 CSIM-set-3-05 no-reuse
    20021030 CSIM-set-3-04 no-reuse
    20021030 CSIM-set-3-03 no-reuse
    20021028 CSIM-set-3-02 no-reuse
    20021023 CSIM-set-3-01 no-reuse
    20021022 CSIM-set-3-00 no-reuse
    0 CSIM-set-2-05 reuse
    
     

And that's it.

If you replace a tape, it means Amanda has failed somewhere, you may have to run amflush to same temporary back-up from disk to tape.

Recovering data

This paragraph has to be written according to the experience we will gain.

Restoring some files

  1. Connect on Amanda server, first as yourself and go to the directory /holding/recover.
  2. use the command sudo amrecover normal to run the recovering tool of Amanda (normal is the name of Amanda configuration).
  3. Choose the client that need recovery with sethost host.
  4. Choose the disk on the client that need recovery with setdisk disk.
  5. Choose the date of the backup to use with setdate date in the format YYYY-MM-DD.
  6. Browse the index and add the files you want to restore.
  7. Extract the files from the tapewith the command extract.
  8. You will have to mount some of the cartridges, follow the instructions.
  9. When it is done, you will find the files in /holding/recover/some/directory; push the file back on the original host and clean the directory /holding/recover.

Restoring the root partition on amanda server itself

Restoring the /usr partition on amanda server itself

Building Amanda

Here are a few points about the way to build Amanda client on a FreeBSD machine.

Client is build directly from the ports in /usr/ports/misc/amanda-client with the following arguments to make:

AMANDA_SERVER=amanda.cs.ait.ac.th AMANDA_GNUTAR_LISTDIR=/usr/local/var/amanda/gnutar-lists AMANDA_USER=amanda AMANDA_GROUP=amanda

Of course group and user amanda should existon the client machine.

On database and ufo, building Amanda client requires that Amanda uses the Perl script /usr/local/bin/mygtar instead of /usr/local/bin/gtar. This script will stop MySQL or Zope before the backup and restart them afterward.

To build Amanda with the script mygtar, edit /usr/poprts/misc/amanda-server/Makefile and replace

CONFIGURE_ARGS= --libexecdir=${PREFIX}/libexec/amanda \
	...
    --with-gnutar=${LOCALBASE}/bin/gtar

with

CONFIGURE_ARGS= --libexecdir=${PREFIX}/libexec/amanda \
	...
    --with-gnutar=${LOCALBASE}/bin/mygtar

You need to edit the script mygtar to specify what disk is concerned by the procedure and what services should be stopped and restarted. Only services valid are those started in /usr/local/etc/rc.d/. The script will make a diagnostic in /tmp/amanda/myruntar.dateandtime.debug.

 
#!/usr/bin/perl -w

# The list of services to stop before the backup and start afterward
# Those services are in /usr/local/etc/rc.d
@services=("mysql-server");

# The DLE concerned
$disk="/database";

# How long we keep the logs? (in sec.)
$old=24*3600*3; # 3 days

use Getopt::Long qw(:config pass_through);

my @saveopts = @ARGV;
GetOptions (
        'create' => \$create,
        'directory=s' => \$dir,
        'file=s' => \$file,
	'listed-incremental=s' => \$incremental,
);
@ARGV = @saveopts;

my $postproc = 0;
if ($create  &&  $dir eq $disk &&  $file ne '/dev/null') {
    $incr=0;
    $incremental=~/_(\d).new$/;
    $incr=1 if $1>0;
    &preproc($incr);
    $postproc = 1;
}

unshift(@ARGV, "/usr/local/bin/tar");
system @ARGV;

my $rc = $? >> 8;

if ($postproc) {
    &postproc($incr);
}

exit $rc;

sub postproc {
    my $incr=$_[0];

    print LOG "Tared\nStarting services\n";
    close LOG;
    foreach $service (@services) {
	system "/usr/local/etc/rc.d/$service.sh start >>$logfile 2>&1"
	    if -f "/usr/local/etc/rc.d/$service.sh";
    }
    open LOG, ">>$logfile";
    print LOG "Started\n";
    close LOG;
}

sub preproc {
    my $incr=$_[0];
    
    # find the log file
    @t=localtime;
    $time=sprintf("%04d%02d%02d%02d%02d%02d", $t[5]+1900, $t[4]+1, $t[3], 
		  $t[2], $t[1], $t[0]);
    $logfile="/tmp/amanda/myruntar.$time.debug";
    open LOG, ">$logfile";
    ($l, $p, $u, $g)=getpwnam "amanda";
    $l=$p; # remove a warning
    chown $u, $g, "/tmp/amanda/myruntar.$time.debug";

    # remove the old tmp/amanda/myruntar logs
    opendir DIR, "/tmp/amanda";
    $now=time;
    while ($dir=readdir DIR) {
	next unless $dir=~/^myruntar.*$/;
	@s=stat "/tmp/amanda/$dir";
	unlink "/tmp/amanda/$dir" if $s[10]< $now-$old;
    }
    close DIR;

    print LOG "Stopping services\n";
    close LOG;

    foreach $service (@services) {
	system "/usr/local/etc/rc.d/$service.sh stop >>$logfile 2>&1"
	    if -f "/usr/local/etc/rc.d/$service.sh";
    }
    open LOG, ">>$logfile";
    print LOG "Stopped\nRunning GNU tar... ";
}
 

CSIM home pageWMailAccount managementCSIM LibraryNetwork test toolsSearch CSIM directories
Contact us: Olivier Nicole CSIM    SET    AIT Last update: Apr 2009