2 Chapter 4. Indexing with AMANDA
3 Prev Part I. Installation Next
5 -------------------------------------------------------------------------------
7 Chapter 4. Indexing with AMANDA
16 XML-conversion;Updates
28 Protocol_Between_amindexd_and_amrecover
34 Changes_from_amindex-1.0
36 Changes_from_amindex-0.3
38 Changes_from_amindex-0.2
40 Changes_from_amindex-0.1
42 Changes/additions_to_2.3.0
49 Refer to http://www.amanda.org/docs/indexing.html for the current version of
51 This file describes how the index files are generated and how amrecover is
56 The database consists of a directory tree of the format: $host/$disk/
58 The host and disk are those listed in the disklist file, the "$host/$disk/" is
59 like the curinfo database, '/' are changed for '_'. There is an index file for
60 each dump, the name of the file is made of the date and the level, they will
61 have the .gz suffix if they are compressed with gzip.
62 ex. The file foo/_usr/19991231_0.gz is the index of the level 0 made on
63 19991231 of the disk /usr of the host foo.
64 The files are ASCII text files containing a list of the directory and files of
65 the dump, one per line. Each entry is the filename relative to the mount point,
66 starting with a /, e.g., /home/user1/data from the disk mounted on /home would
67 generate the entry /user1/data. The index files are stored in compressed format
68 (eg gzip or compress).
72 The client is called amrecover and is loosely based on the functionality of the
73 program recover from Backup Copilot. A user starts up amrecover. This requires
74 specifying the index server and the AMANDA config name (defaults for both are
75 compiled in as part of the installation). Then the user has to specify the name
76 of the host information is wanted about, the disk name, and (optionally) the
77 disk mount point. Finally a date needs to be specified. Given all this, the
78 user can then roam around a virtual file system using ls and cd much like in a
79 FTP client. The file system contains all files backed up on the specified date,
80 or before that date, back to the last level 0 backup. Only the most recent
81 version of any file is shown.
82 As the file system is traversed, the user can add and delete files to a
83 "shopping list", and print the list out.
87 When a user has built up a list of files to extract, they can be extracted by
88 issuing the command extract within amrecover.
89 Files are extracted by the following, for each different tape needed.
90 As part of the installation, a "tape server" daemon amidxtaped is installed on
91 one or more designated hosts, which have an attached tape drive. This is used
92 to read the tapes. See the config files for the options for specifying a
94 amrecover contacts amidxtaped on the tape server host specifying which tape
95 device to use, which host and disk files are needed for. On the tape server
96 host, amidxtaped executes amrestore to get the dump image file off the tape,
97 and returns the data to amrecover.
98 If dumps are stored compressed for the client, then amrecover pipes the data
99 through the appropriate uncompression routine to uncompress it before piping it
100 into restore, which then extracts the required files from the dump image.
101 Note that a user can only extract files from a host running the same operating
102 system as he/she is executing amrecover on, since the native dump/restore tools
103 are used - unless GNU-tar is used.
105 Protocol Between amindexd and amrecover
107 The protocol talked between amindexd and amrecover is a simple ASCII chat
108 protocol based on that used in FTP. amrecover sends a 1 line command, and
109 amindexd replies with a 1 line or multi-line reply. Each line of the reply
110 starts with a three digit code, starting with a '5' if an error occurred. For 1
111 line replies, and the last line of a multi-line reply, the 4th character is a
112 space. For all but the last line of a multi-line reply, the 4th character is a
114 The commands and replies other than acknowledgments are:
115 Table 4.1. Protocol between amindexd and amrecover
116 ____________________________________________________________________________
117 |QUIT_______________|finish_up_and_close_connection__________________________|
118 |HOST_<host>________|set_host_to_host________________________________________|
119 |DISK_<disk>________|set_disk_to_disk________________________________________|
120 |LISTDISK_[<device>]|list_the_disks_for_the_current_host_____________________|
121 |SCNF_<config>______|set_AMANDA_configuration_to_config______________________|
122 |DATE_<date>________|set_date_to_date________________________________________|
123 |DHST_______________|return_dump_history_of_current_disk_____________________|
124 | |Opaque is directory? query. Is the directory dir present|
125 |OISD <dir> |in the backups of the current disk back to and including|
126 |___________________|the_last_level_0_dump.__________________________________|
127 | |Opaque list directory. Give all filenames present in dir|
128 |OLSD <dir> |in the backups of the current disk back to and including|
129 |___________________|the_last_level_0_dump.__________________________________|
130 | |Opaque recursive list directory. Give all filenames |
131 |ORLD <dir> |present in dir and subdir in the backups of the current |
132 |___________________|disk_back_to_and_including_the_last_level_0_dump._______|
133 |TAPE_______________|return_value_of_tapedev_from_amanda.conf_if_set.________|
134 |DCMP |returns "YES" if dumps for disk are compressed, "NO" if |
135 |___________________|dumps_aren't.___________________________________________|
141 1. Whether or not an index is created for a disk is controlled by a disk
142 configuration option index. So, in amanda.conf you need to define a
143 disktype with this option, e.g.,
145 define dumptype comp-user-index {
146 comment "Non-root partitions on reasonably fast machines"
152 2. You need to define disks that you want to generate an index for to
153 be of one of the disktypes you defined which contain the index option.
154 This cause sendbackup-dump on the client machine to generate an index file
155 which is stored local to the client, for later recovery by amgetidx (which
156 is called by amdump).
157 3. AMANDA saves all the index files under a directory specified by
158 "indexdir" in amanda.conf. You need to create this directory by hand. It
159 needs to have read/write permissions set for the user you defined to run
161 If you are using the "text database" option you may set indexdir and
162 infofile to be the same directory.
163 4. The index browser, amrecover, currently gets installed as part of the
164 client software. Its location may not be appropriate for your system and
165 you may need to move it to a more accessible place such as /usr/local/bin.
166 See its man page for how to use it.
167 Note that amindexd, amgetidx, amidxtaped, and amtrmidx all write debug
168 files on the server in /tmp (unless this feature is disabled in the source
169 code), which are useful for diagnosing problems. amrecover writes a debug
170 file in /tmp on the machine it is invoked.
175 The userid chosen to run the AMANDA client code must have permission to run
176 restore since this is used by createindex-dump to generate the index files.
177 For a user to be able to restore files from within amrecover, that user must
178 have permission to run restore.
180 Changes from amindex-1.0
182 Get index directory from amanda.conf.
183 Integration into AMANDA-2.3.0.4.
184 Rewriting of amgetidx to use amandad instead of using rsh/rcp.
186 Changes from amindex-0.3
188 Support for index generation using GNU-tar.
189 Support for restoring files from within amrecover.
192 * index/client/amrecover.c (guess_disk): Removed inclusion of mntent.h and use
193 of MAXMNTSTR since this was non-portable, as pointed out by Izzy Ergas
197 * index/client/display_commands.c (list_directory): Removed point where
198 list_directory() could sleep for ever waiting for input that wasn't going to
202 * index/server/amindexd.c index/client/uscan.l Installed patches from Les
203 Gondor <les@trigraph.on.ca> to make amrecover handle spaces in file names.
206 * server-src/amcontrol.sh: As pointed out by Neal Becker <neal@ctd.comsat.com>
207 there were still a few sh-style comments that needed conversion to c-style.
210 Changes from amindex-0.2
213 * index/client/Makefile.in
214 * index/client/help.c
215 * index/client/amrecover.h
216 * index/client/uparse.y
217 * index/client/uscan.l Added a help command.
220 * index/client/set_commands.c: set_disk() and set_host() now check for empty
224 * index/client/extract_list.c:
225 * index/client/amrecover.h:
226 * index/client/uparse.y:
227 * index/client/uscan.l: Added clear extract list command.
230 * index/client/set_commands.c (set_disk): Added code so working directory set
234 * index/client/extract_list.c: If the last item on a tape list is deleted, the
235 tape list itself is now deleted from the extract list.
238 * index/client/amrecover.c:
239 * index/server/amindex.c: If the server started up and found that the index dir
240 doesn't exist, then it exited immediately and the client got informative
241 message. Corrected this so it is obvious what is wrong to the user, since
242 this is most likely to occur when somebody is setting up for the first time
243 and needs all the help they can get.
246 * server-src/amgetidx.c Added patch from Pete Geenhuizen
247 <pete@gasbuggy.rockledge.fl.us> so that it works even when remote shell is
251 * server-src/amcontrol.sh
252 * server-src/Makefile.in Amcontrol is now parameterized like other scripts and
253 run through munge to generate installable version.
256 * index/server/amindexd.c (main): Added code to set userid if FORCE_USERID set.
259 * index/server/amindexd.c Removed #define for full path of grep. Assumed now to
263 * client-src/createindex-dump.c
264 * client-src/sendbackup-dump.c
265 * man/Makefile.in Added patch from Philippe Charnier <charnier@lirmm.fr> so
266 they work when things are installed with version numbers. This was also
267 reported by Neal Becker <neal@ctd.comsat.com>. Also patch to set installed
268 man page modes and create directory if needed.
271 * config/options.h-sunos4 Corrected definition for flex library.
274 * server-src/amtrmidx.c Added some pclose() commands, used remove() instead of
275 system("rm .."). Problems reported by Pete Geenhuizen
276 (<pete@gasbuggy.rockledge.fl.us>) on a system with small ulimits set.
279 * index/server/amindexd.[ch]
280 * index/server/list_dir.c
281 * index/client/amrecover.c
282 * index/client/set_commands.c
283 * index/client/uparse.y Changes developed with the help of Pete Geenhuizen
284 <pete@gasbuggy.rockledge.fl.us> to support disks specified by logical names.
285 Also, now debug files generated by amrecover include PID so multiple users
286 can use amrecover simultaneously and without file deletion permission
290 * config/config.h-hpux:
291 * config/config-common.h:
292 * server-src/amgetidx.c: Changes from Neal Becker re remote shell, making it a
293 configuration parameter.
296 * config/options.h-sunos4 Had -Lfl instead of -lfl
299 Changes from amindex-0.1
302 * index/client/uscan.l: added support for abbreviated date specs
305 * index/client/amrecover.c (guess_disk): guess_disk got disk_path wrong if
306 mount point other than / (as subsequently pointed out by Eir Doutreleau
310 * server-src/amtrmidx: Added amtrmidx which removes old index files.
313 * index/client: Added a pwd command
316 * server-src/amgetidx.c (main): Added use of CLIENT_LOGIN username on r
317 commands. (as pointed out by Eric Payan <Eric.Payan@ufrima.imag.fr>)
320 * server-src/amgetidx.c: Bug: It was copying from all clients irrespective of
321 whether the client was configured for indices. A '}' in the wrong place.
324 * server-src/amgetidx.c: Removed user configuration section. Instead include
325 amindexd.h to get information.
328 Changes/additions to 2.3.0
330 common-src/conffile.[ch]
332 * added "index" as a valid option
334 server-src/driverio.c
336 * added code to optionstr() to write "index" into option string
338 client-src/sendback-dump.c
340 * added code to generate index if requested.
342 client-src/indexfilename.[ch] client-src/createindex-dump.c
344 * code to generate index.
346 client-src/Makefile.in
348 * a new target. Another file for sendbackup-dump
350 config/config-common.h
352 * added def of restore.
358 * Empty directories don't get into the listing for a dump (at all dump levels).
361 * When amrecover starts up, it tries to guess the disk and mount point from the
362 current directory of the working system. This doesn't work for disks
363 specified by logical names, nor when an automounter is being used, or a link
366 -------------------------------------------------------------------------------
369 Chapter 3. Excluding Home Chapter 5. Backup PC hosts using Samba