X-Git-Url: https://git.gag.com/?a=blobdiff_plain;f=docs%2Fdumperapi.txt;h=e69de29bb2d1d6434b8b29ae775ad8c2e48c5391;hb=d92f70685083588e2a7ce6bc312a735f6937b5a6;hp=770e1d585a9dfa6b849f2dddc3933203e4692c30;hpb=cdbbeef9cde260e429854dd313bc0bf7560e1e24;p=debian%2Famanda diff --git a/docs/dumperapi.txt b/docs/dumperapi.txt index 770e1d5..e69de29 100644 --- a/docs/dumperapi.txt +++ b/docs/dumperapi.txt @@ -1,325 +0,0 @@ - - Chapter 23. Amanda dumper API -Prev Part V. Technical Background Next - -------------------------------------------------------------------------------- - -Chapter 23. Amanda dumper API - - -Alexandre Oliva - -Original text -AMANDA Core Team -> - -Stefan G. Weichinger - -XML-conversion;Updates -AMANDA Core Team - -Table of Contents - - - Introduction - - The_Problem - - Overview_of_the_API - - - The_`support'_command - - - The_`selfcheck'_command - - The_`estimate'_and_`estimate-parse'_commands - - The_`backup'_and_`backup-parse'_commands - - The_`index-from-output'_and_`index-from-image'_commands - - The_`restore'_command - - The_`print-command'_command - - Conclusion - - -Note - -Refer to http://www.amanda.org/docs/dumperapi.html for the current version of -this document. - - Introduction - -This is a proposal of a mechanism for Amanda to support arbitrary backup -programs, that relies on a generic backup driver and scripts or programs that -interface with backup programs such as dump, tar, smbclient, and others. It can -also be used to introduce pre- and post-backup commands. -The interface is simple, but supports everything that is currently supported by -Amanda, and it can be consistently extended to support new abstractions that -may be introduced in the backup driver in the future. -This proposal does not imply any modification in the Amanda protocol or in -Amanda servers; only Amanda clients have to be modified. By Amanda clients, we -refer to hosts whose disks are to be backed up; an Amanda server is a host -connected to a tape unit. -Currently (as of release 2.4.1 of Amanda), Amanda clients support three -operations: selfcheck, estimate and backup. -Selfcheck is used by the server program amcheck, to check whether a client is -responding or if there are configuration or permission problems in the client -that might prevent the backup from taking place. -Estimates are requested by the Amanda planner, that runs on the server and -collects information about the expected sizes of backups of each disk at -several levels. Given this information and the amount of available tape space, -the planner can select which disks and which levels it should tell dumper to -run. -dumper is yet another server-side program; it requests clients to perform -dumps, as determined by planner, and stores these dumps in holding disks or -sends them directly to the taper program. The interaction between dumper and -taper is beyond the scope of this text. -We are going to focus on the interaction between the Amanda client program and -wrappers of dump programs. These wrappers must implement the DUMPER API. The -dumptype option `program' should name the wrapper that will be used to back up -filesystems of that dumptype. One wrapper may call another, so as to extend its -functionality. - - The Problem - -Different backup programs present distinct requirements; some must be run as -super-user, whereas others can be run under other user-ids. Some require a -directory name, the root of the tree to be backed up; others prefer a raw -device name; some don't even refer to local disks (SAMBA). Some wrappers may -need to know a filesystem type in order to decide which particular backup -program to use (dump, vdump, vxdump, xfsdump, backup). -Some provide special options for estimates, whereas others must be started as -if a complete dump were to be performed, and must be killed as soon as they -print an estimate. -Furthermore, the output formats of these backup programs vary wildly. Some will -print estimates and total sizes in bytes, in 512-byte tape blocks units, in -Kbytes, Mbytes, Gbytes, and possibly Tbytes in the near future. Some will print -a timestamp for the backup; some won't. -There are also restrictions related with possible scheduling policies. For -example, some backup programs only support full backups or incrementals based -on the last full backup (0-1). Some support full backups or incrementals based -on the last backup, be it a full or an incremental backup (0-inf++). Some -support incrementals based on a timestamp (incr/date); whereas others are based -on a limited number of incremental levels, but incrementals of the same level -can be repeated, such as dump (0-9). -Amanda was originally built upon DUMP incremental levels, so this is the only -model it currently supports. Backup programs that use other incremental -management mechanisms had to be adapted to this policy. Wrapper scripts are -responsible for this adaptation. -Another important issue has to do with index generation. Some backup programs -can generate indexes, but each one lists files in its own particular format, -but they must be stored in a common format, so that the Amanda server can -manipulate them. -The DUMPER API must accomodate for all these variations. - - Overview of the API - -We are going to define a standard format of argument lists that the backup -driver will provide to wrapper programs, and the expected result of the -execution of these wrappers. -The first argument to a wrapper should always be a command name. If no -arguments are given, or an unsupported command is requested, an error message -should be printed to stderr, and the program should terminate with exit status -1. - - The `support' command - -As a general mechanism for Amanda to probe for features provided by a backup -program, a wrapper script must support at least the `support' command. Some -features must be supported, and Amanda won't ever ask about them. Others will -be considered as extensions, and Amanda will ask the wrapper whether they are -supported before issuing the corresponding commands. - - The `level-incrementals' subcommand - -For example, before requesting for an incremental backup of a given level, -Amanda should ask the wrapper whether the backup program supports level-based -incrementals. We don't currently support backup programs that don't, but we may -in the future, so it would be nice if wrappers already implemented the command -`support level-incrementals', by returning a 0 exit status, printing, say, the -maximum incremental level it supports, i.e., 9. A sample session would be: - - % /usr/local/amanda/libexec/wrappers/dump support level-incrementals hda0 9 - - -Note that the result of this support command may depend on filesystem -information, so the disklist filesystem entry should be specified as a command -line argument. In the next examples, we are not going to use full pathnames to -wrapper scripts any more. -We could have defined a `support' command for full backups, but I can't think -of a backup program that does not support full backups... - - The `index' subcommand - -The ability to produce index files is also subject to an invocation of -`support' command. When the support sub-command is `index', like in the -invocation below, the wrapper must print a list of valid indexing mechanisms, -one per line, most preferred first. If indexing is not supported, nothing -should be printed, and the exit status should be 1. -DUMP support index hda0 -The currently known indexing mechanisms are: -output: implies that the command `index-from-output' generates an index file -from the output produced by the backup program (for example, from tar -cv). -image: implies that the command `index-from-image' generates an index file from -a backup image (for example, tar -t). -direct: implies that the `backup' command can produce an index file as it -generates the backup image. -parse: implies that the `backup-parse' command can produce an index file as it -generates the backup formatted output . -The indexing mechanisms will be explicitly requested with the additionnal -option `index-' in the `backup' and `backup-parse' command invocation. -`index-from-image' should be supported, if possible, even if other index -commands are not, since it can be used in the future to create index files from -previously backed up filesystems. - - The `parse-estimate' subcommand - -The `parse-estimate' support subcommand print a list of valid mechanisms to -parse the estimate output and write the estimate size to its output, the two -mechanisms are: -direct: implies that the `estimate' command can produce the estimate output. -parse: implies that the `estimate-parse' command can produce the estimate -output when fed with the `estimate' output. -The estimate parsing mechanisms will be explicitly requested with the -additionnal option `estimate-' in the `estimate' and `estimate-parse' -command invocation. - - The `parse-backup' subcommand - -The `parse-backup' support subcommand print a list of valid mechanisms to parse -the backup stderr, the two mechanisms are: -direct: implies that the `backup' command can produce the backup-formatted- -ouput. -parse: implies that the `backup-parse' command can produce the backup- -formatted-ouput when fed with the `backup' stderr. -The backup parsing mechanisms will be explicitly requested with the additional -option `backup-' in the `backup' and `backup-parse' command invocation. - - Other subcommands - -Some other standard `support' sub-commands are `exclude' and `exclude-list'. -One may think (and several people did :-) ) that there should be only one -support command, that would print information about all supported commands. The -main arguments against this proposal have to do with extensibility: -The availability of commands might vary from filesystem to filesystem. No, I -don't have an example, I just want to keep it as open as possible :-) one -support subcommand may require command line arguments that others don't, and we -can't know in advance what these command line arguments are going to be -The output format and exit status conventions of a support command may vary -from command to command; the only pre-defined convention is that, if a wrapper -does not know about a support subcommand, it should return exit status 1, -implying that the inquired feature is not supported. - - The `selfcheck' command - -We should support commands to perform self-checks, run estimates, backups and -restores (for future extensions of the Amanda protocol so as to support -restores) -A selfcheck request would go like this: -DUMP selfcheck hda0 option option=value ... -The options specified as command-line arguments are dumptype options enabled -for that disk, such as `index', `norecord', etc. Unknown options should be -ignored. For each successful check, a message such as: -OK [/dev/hda0 is readable] OK [/usr/sbin/dump is executable] -Errors should be printed as: -ERROR [/etc/dumpdates is not writable] -A wrapper script will certainly have to figure out either the disk device name -or its mount point, given a filesystem name such as `hda0', as specified in the -disklist. In order to help these scripts, Amanda provides a helper program that -can guess device names, mount points and filesystem types, when given disklist -entries. -The filesystem type can be useful on some operation systems, in which more than -one dump program is available; this information can help automatically -selecting the appropriate dump program. -The exit status of selfcheck and of this alternate script are probably going to -be disregarded. Anyway, for consistency, selfcheck should return exit status 0 -for complete success, 1 if any failures have occurred. - - The `estimate' and `estimate-parse' commands - -Estimate requests can be on several different forms. An estimate of a full -backup may be requested, or estimates for level- or timestamp-based -incrementals: -DUMP estimate full hda0 option ... DUMP estimate level 1 hda0 option ... DUMP -estimate diff 1998:09:24:01:02:03 hda0 option ... -If requested estimate type is not supported, exit status 3 should be returned. -If the option `estimate-direct' is set, then the `estimate' command should -write to stdout the estimated size, in bytes, a pair of numbers that, -multiplied by one another, yield the estimated size in bytes. -If the option `estimate-parse' is set, then the `estimate' command should write -to stdout the informations needed by the `estimate-parse' command, that should -extract from its input the estimated size. -The syntax of `estimate-parse' is identical to that of `estimate'. -Both `estimate' and `estimate-parse' can output the word `KILL', after printing -the estimate. In this case, Amanda will send a SIGTERM signal to the process -group of the `estimate' process. If it does not die within a few seconds, a -SIGKILL will be issued. -If `estimate' or `estimate-parse' succeed, they should exit 0, otherwise exit -1, except for the already listed cases of exit status 3. - - The `backup' and `backup-parse' commands - -The syntax of `backup' is the same as that of `estimate'. The backup image -should be written to standard output, whereas stderr should be used for the -user-oriented output of the backup program and other messages. -If the option `backup-direct' is set, then the `backup' command should write to -stderr a formatted-output-backup. -If the option `backup-parse' is set, then the `backup' command should write to -stderr the informations needed by the `backup-parse' command, that should edit -its input so that it prints to standard output a formatted-output-backup. -If the option `no-record' is set, then the `backup' command should not modify -its state file (ex. dump should not modify /etc/dumpdates). -The syntax of `backup-parse' is identical to that of `backup'. -The syntax of the formatted-output-backup is as follow: All lines should start -with either `| ' for normal output, `? ' for strange output or `& ' for error -output. If the wrapper can determine the total backup size from the output of -the backup program, it should print a line starting with `# ', followed by the -total backup size in bytes or by a pair of numbers that, multiplied, yield the -total backup size; this number will be used for consistency check. -The option `index-direct' should cause commands `backup' to output the index -directly to file descriptor 3. The option `index-parse' should cause commands -`backup-parse' to output the index directly to file descriptor 3. The syntax of -the index file is described in the next section. - - The `index-from-output' and `index-from-image' commands - -The syntax of the `index-from-output' and `index-from-image' commands is -identical to the one of `backup'. They are fed the backup output or image, and -they must produce a list of files and directories, one per line, to the -standard output. Directories must be identified by the `/' termination. -After the file name and a blank space, any additional information about the -file or directory, such as permission data, size, etc, can be added. For this -reason, blanks and backslashes within filenames should be quoted with -backslashes. Linefeeds should be represented as `\n', although it is not always -possible to distinguish linefeeds in the middle of filenames from ones that -separate one file from another, in the output of, say `restore -t'. It is not -clear whether we should also support quoting mechanisms such as `\xHH', `\OOO' -or `\uXXXX'. - - The `restore' command - -Yet to be specified. - - The `print-command' command - -This command must be followed by a valid backup or restore command, and it -should print a shell-command that would produce an equivalent result, i.e., -that would perform the backup to standard output, or that would restore the -whole filesystem reading from standard input. This command is to be included in -the header of backup images, to ease crash-recovery. - - Conclusion - -Well, that's all. Drop us a note at the amanda-hackers mailing list (mailto:// -amanda-hackers@amanda.org) if you have suggestions to improve this document -and/or the API. Some help on its implementation would be welcome, too. -------------------------------------------------------------------------------- - -Prev Up Next -Chapter 22. How Amanda uses UDP and TCP Home Chapter 24. Amanda Internals -ports -