1 This is an attempt to document Amanda's internals. Please feel free to make
2 comments and suggest changes. Text for new sections gratefully accepted!
5 (George.Scott@cc.monash.edu.au)
12 Client I Server +-planner-+
16 +-amandad-+ I | | (______)
18 | | I | | all ::::>| |
25 | | I +-driver--+ | log |
26 +---------+ I | si | all ::::>| |
29 +-sendback+ I | | +------+
31 | | I ::::::::| |<::::::
33 | | I :: ::::>| |:::: ::
35 | | I :: :: +---------+ :: ::
38 | | I :: :: ______ :: ::
39 +---------+ I :: :: (______) :: ::
40 I :: :: | /tmp | :: ::
42 +-dump----+ I +-dumper--+ :: +------+ +-taper/r-+ +-taper/w-+
43 | | I | si so | :: | si so | | |
44 | | I | | :: ______ | | | |
45 | | I |mesgfd | :: (______) | | | |
46 | se|::::>|::::::::>|:: | hold | | | | |
47 | | I | errf| | disk | | p2c|:::>|p2c |
48 | | I | | | | | c2p|<:::|c2p | ____
49 | | I |datafd | ::>| |::: |fd | | | / \
50 | so|::::>|::::::::>|:: +------+ :>|::::::::>SHDMEM::::::::>|::>|tape|
51 | | I | outfd| :: :: | | | tapefd| \ _ /
52 +---------+ I +---------+ :::::::::::: +---------+ +---------+
56 server and amandad on client
57 ============================
59 XXX - still to be done
65 Planner interrogates all clients and generates a plan of which disks to
66 backup and what dump level to do them at. The plan is plain text with one
67 line per disk to be dumped. It is piped from planners stdout to drivers
68 stdin. Plan lines come in two flavours:
71 <host> <disk> <pri> <lev> <size> <time> <deg lev> <deg size> <deg time>
73 For incremental dumps:
74 <host> <disk> <pri> <lev> <size> <time>
77 <host> Host name of client (from disklist file)
78 <disk> Name of disk (from disklist file)
79 <pri> Priority of backup (pri from disklist and amanda.conf + days
81 <lev> Dump level for dump (0 for total, 1-9 for incremental)
82 <size> Estimated size (in Kb after compression if requested)
83 <time> Estimated time for backup (in seconds)
84 <deg lev> <lev> to use if in degraded mode
85 <deg size> <size> to use if in degraded mode
86 <deg time> <time> to use if in degraded mode
92 Driver talks via two pipes connected to each dumper's stdin and stdout. The
93 commands and responses are plain text.
95 Driver can ask dumper to do a dump to a file on the holding disk:
96 FILE-DUMP <handle> <filename> <host> <disk> <level> <dumpdate> <chunksize> <prog> <use> <options>
98 PORT-DUMP <handle> <port> <host> <disk> <level> <dumpdate> <prog> <options>
99 or exit at the end of the run:
102 If the dump finishes correctly dumper replies with:
103 DONE <handle> [<message>]
105 If something goes wrong with the dump, dumper can request that the dump be
106 retried at a later time with:
107 TRY-AGAIN <handle> [<message>]
108 or, for fatal errors, be abandoned with:
109 FAILED <handle> [<message>]
111 If the holding disk runs out of space (i. e. 'disk full'), dumper will give:
112 NO-ROOM <handle> <missing_size>
113 After that, it will issue RQ_MORE_DISK (see below). The driver will correct
114 its notion of free diskspace by subtracting <missing_size> from the
115 holding disk's alleged size.
117 If the dumper has used up all of its allocated holding disk space, it will
119 RQ-MORE-DISK <handle>
120 and wait for driver to either allocate more diskspace for the dumper and say
121 CONTINUE <filename> <chunksize> <use>
126 If driver says something that dumper doesn't recognise it responds with:
127 BAD-COMMAND <message>
131 <filename> Name of file on holding disk to write dump (see the section
132 'Multiple Holding Disks' below)
133 <port> Port (of taper) to send dump directly
134 <host> Hostname of client
135 <disk> Disk to backup
136 <level> Dump level to do backup at
137 <prog> Dump program to use
138 <options> Options to pass to sendbackup
139 <message> Error or status message
140 <use> Amount of diskspace to use on the holding disk given by
142 <chunksize> Maximum size of a file on the given holding disk
143 <missing_size> Difference between the allocated space and the space
144 actually used by a dumper when a DISK FULL occurs
150 Driver talks via two pipes connected to taper's stdin and stdout. The
151 commands and responses are plain text.
153 Driver initialises taper with:
154 START-TAPER <datestamp>
155 to which taper replies with:
157 or, for fatal errors, with:
158 TAPER-ERROR [<message>]
160 Driver can ask taper to to copy a file from the holding disk to tape:
161 FILE-WRITE <handle> <filename> <host> <disk> <level>
162 or directly from a dumper:
163 PORT-WRITE <handle> <host> <disk> <level>
164 or exit at the end of the run:
167 Taper responds to the PORT-WRITE command with:
169 which driver should then hand on to dumper in a PORT-DUMP command.
171 Taper responds to the QUIT command with:
174 If the copy to tape finishes correctly taper replies with:
175 DONE <handle> [<message>]
177 If something goes wrong with the tape, taper can request that the dump be
178 retried at a later time with:
179 TRY-AGAIN <handle> [<message>]
180 or, for fatal errors, be abandoned with:
181 TAPE-ERROR <handle> [<message>]
183 If driver says something that taper doesn't recognise it responds with:
184 BAD-COMMAND <message>
187 <datestamp> Todays date as "yymmdd"
189 <filename> Name of file (on holding disk) to write dump
190 <port> Port (of taper) to send dump directly
191 <host> Hostname of client
192 <disk> Disk to backup
193 <level> Dump level to do backup at
194 <message> Error or status message
197 taper(read) and taper(write)
198 ============================
200 There are two parts to taper: the file reader and the tape writer.
201 Communication between the two sides is via a bit of shared memory for data
202 transfer and two pipes (one in each direction) for synchronisation.
204 The shared memory area is made up of NBUFS (=20) buffers each of which
205 contains a status word and a BUFFER_SIZE (=32*1024) byte data buffer.
207 The sync pipes are used to transfer a simplistic command sequence:
212 Startup S<datestamp> --->
216 Open tape O<datestamp><hostname><diskname><level>
220 Write buffer W<bufnum> --->
221 <--- R<bufnum> Buffer empty
222 <--- E<message> Error
223 <--- T<message> Error, try again
225 Protocol error X --->
229 <--- C<label><filenum><stats> Closing
234 Multiple Holding Disks
235 ======================
237 (This section was written by Peter Conrad <conrad@opus5.de>)
239 Since amanda version x.x.x a single dump file may be spread across multiple
240 holding disks. Some changes in the driver-dumper protocol have been introduced
241 for this, as well as in the allocation of holding diskspace.
243 1. Before the dump starts, an initial amount of holding diskspace is
244 allocated for the dump. The amount is based on the estimate for the
246 2. If there is enough space available on the holding disks a list of holding
247 disks is created. For each holding disk to be used by the dumper, the
248 space to be used by this dumper and the chunksize of that holding disk
249 are noted (and, of course, the name of the holding disk).
250 3. This list is turned into an array? of structures containing the values
251 found in step 2. The first set of values is sent to the dumper.
252 4. A dumper is responsible for sticking to the given values:
253 - no file created by the dumper may be larger than the corresponding
255 - on any given holding disk, no more than 'use' KBytes of diskspace may
257 5. a) When all allocated diskspace has been used and the dump still hasn't
258 finished, the dumper may request more space from the driver. The driver
259 can either send a filename and additional size to be used, possibly on
260 a different hodling disk, or tell the dumper to abort the dump.
261 b) When the disk currently in use by the dumper runs full the dumper
262 will tell the driver about this situation. The driver will react like
263 in step a), but it will correct its notion of free diskspace.
264 6. If the driver supplies another list of holding disks (not necessarily as
265 much as the dumper requested), the dumper interprets the given 'use' values
266 as *additional* diskspace it may use.
269 The protocol between driver and taper didn't need to be modified, because
270 each chunk of a dump contains the full pathname of the succeeding chunk.