2 * Copyright (c) 2009, 2010 Zmanda, Inc. All Rights Reserved.
4 * This program is free software; you can redistribute it and/or modify it
5 * under the terms of the GNU General Public License version 2 as published
6 * by the Free Software Foundation.
8 * This program is distributed in the hope that it will be useful, but
9 * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
10 * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
13 * You should have received a copy of the GNU General Public License along
14 * with this program; if not, write to the Free Software Foundation, Inc.,
15 * 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
17 * Contact information: Zmanda Inc., 465 S. Mathilda Ave., Suite 300
18 * Sunnyvale, CA 94085, USA, or: http://www.zmanda.com
25 Amanda::Device - interact with Amanda data-storage devices
29 use Amanda::Device qw( :constants );
31 my $dev = Amanda::Device->new($device_name);
32 if ($dev->read_label() == $DEVICE_STATUS_SUCCESS) {
33 print "Label on $device_name is '$dev->volume_label'\n";
38 A volume is a container for data which can be "loaded" into a particular
39 device. For tape devices, a volume is a tape, but most other devices do not
40 deal with such physical objects. Each volume has a volume header giving, among
41 other things, the label of the volume and the timestamp on which it was
42 written. The header may also indicate that the volume is not an Amanda volume.
43 Aside from the header, a volume contains a sequence of files, numbered starting
44 at 1. While writing, devices number files sequentially, but devices that
45 support partial volume recycling may have "holes" in the sequence of file
46 numbers where files have been deleted. The C<seek_file> method, below,
47 describes how the API represents this situation. Each file has a header, too,
48 which contains lots of information about the file. See L<Amanda::Header> for
49 the full list. After the header, a file is just a sequence of bytes.
51 Reads and writes to devices take place in blocks. Unlike a typical
52 operating-system file, in which any block boundaries are lost after the file is
53 written, devices must be read back with the block sizes that were used to read.
54 See C<amanda-devices(7)> for more in block sizes, and the read_block and
55 write_block sections, below, for more information.
57 =head1 USING THE DEVICE API
59 The Device API is object-oriented, so the first task in using the API is to
62 $dev = Amanda::Device->new("tape:/dev/nst0");
64 This function takes a device name (possibly a device alias) and returns a
65 device object. This function always returns a Device, although it may be a Null
66 device with an error condition. Any C<new> call should be followed by a check
67 of the device's status:
69 $dev = Amanda::Device->new($device_name);
70 if ($dev->status() != $Amanda::Device::DEVICE_STATUS_SUCCESS) {
71 die "Could not open '$device_name': " . $dev->error();
74 This function does not access the underlying hardware or any other external
75 systems in any way: that doesn't happen until C<read_label> or C<start>. An
76 Amanda configuration must be loaded when this function is called, as it
77 searches the configuation for device definitions. The member variable
78 C<device_name> is set when this function has returned.
80 It is unusual for higher-level code to call C<< Amanda::Device->new >>.
81 Intead, use L<Amanda::Changer> to load a volume and reserve access to it; the
82 resulting reservation will contain an already-created Device object.
84 While Amanda proivdes multiple implementations of the Device class, they are
85 not distinguishable via the usual Perl methods (C<ref> or C<< $dev->isa >>).
87 Device users generally call device methods in the following order for reading:
94 or, when writing or appending:
99 write_block (repeated)
103 =head2 Alternate Constructor
105 To create a new RAIT device from a collection of device objects, call
106 C<< Amanda::Device->new_rait_from_children($child1, $child2, ..) >>.
107 If one of the child objects is C<undef>, the resulting RAIT device
108 will operate in degraded mode.
110 =head2 Error Handling
112 Device methods return a particular value to signal the presence of an error
113 condition. In many cases, this is simply false (exceptions are listed below).
115 When a device signals an error, C<< $dev->status >> and C<< $dev->error >>
116 contain details of what went wrong. Status is a bitfield where each bit that is
117 set indicates a possible problem. Unfortunately, some devices are unable to
118 distinguish states due to limitations of an external system. For example, the
119 tape device often cannot distinguish an empty drive
120 (C<$DEVICE_STATUS_VOLUME_MISSING>) from a hard device error
121 (C<$DEVICE_STATUS_DEVICE_ERROR>), leading to the status
122 C<$DEVICE_STATUS_VOLUME_MISSING>|C<$DEVICE_STATUS_DEVICE_ERROR>. To be clear:
123 as few as one of the status bits may represent a actual problem. If
124 C<$DEVICE_STATUS_VOLUME_UNLABELED> is set along with other bits, it is I<not>
125 safe to assume that an unlabeled volume is available. However, if the bit is
126 not set, then it is safe to assume there is no unlabeled volume present.
130 =item C<$DEVICE_STATUS_SUCCESS>
134 =item C<$DEVICE_STATUS_DEVICE_ERROR>
136 The device is in an unresolvable error state, and further retries are unlikely
139 =item C<$DEVICE_STATUS_DEVICE_BUSY>
141 The device is in use, and should be retried later
143 =item C<$DEVICE_STATUS_VOLUME_MISSING>
145 The device itself is OK, but has no media loaded. This may change if media is
146 loaded by the user or a changer
148 =item C<$DEVICE_STATUS_VOLUME_UNLABELED>
150 The device is OK and media is laoded, but there is no Amanda header or an
151 invalid header on the media.
153 =item C<$DEVICE_STATUS_VOLUME_ERROR>
155 The device is OK, but there was an unresolvable error loading the header from
156 the media, so subsequent reads or writes will probably fail.
160 At the risk of being repetitive, never test a device's status with C<==>,
161 unless it is to C<$DEVICE_STATUS_SUCCESS>. Furthermore, never try to parse the
162 device error messages -- they are only for user consumption, and may differ
163 from device to device.
165 In addition to the status bitfield, a Device also provides a
166 user-comprehensible error message, available from the methods C<error>
167 (returning the error message), C<status_error> (returning the string form of
168 the status), or C<status_or_error> (returning the error message if one is set,
169 otherwise the string form of the status). None of these functions will ever
174 Device properties provide a bidirectional means of communication between
175 devices and their users. A device provides values for some properties, which
176 other parts of Amanda can use to adjust their behavior to suit the device. For
177 example, Amanda will only attempt to append to a volume if the device's
178 properties indicate that it supports this activity. Some devices have
179 additional properties that can be set to control its activity. For example, the
180 S3 Device requires that the users' keys be given via properties.
182 See C<amanda-devices(7)> for more information on device properties and their
185 The methods C<property_get> and C<property_set> are used to get and set
186 properties, respectively. If the indicated property simply does not exist,
187 these functions return an error indication (FALSE), but the device's status
188 remains C<$DEVICE_STATUS_SUCCESS>. If a more serious error occurs, then the
189 device's status is set appropriately.
191 Device properties are easy to handle, as the Perl-to-C glue takes care of all
192 necessary type conversions:
194 $success = $device->property_set("BLOCK_SIZE", $blocksize);
195 $blocksize = $device->property_get("BLOCK_SIZE");
197 If C<property_get> is called in an array context, it returns the property
198 value, its surety, and its source, in that order. If there is an error
199 fetching the property, C<property_get> returns C<undef>.
201 The C<property_list()> method returns a list of all properties:
203 my @props = $device->property_list();
205 its return is an array of hashes:
207 ( { 'access' => $access_flags,
208 'name' => $property_name,
209 'description' => $property_description },
213 =head3 Surety and Source
215 All properties have a source - where the value came from - and surety - a level
216 of confidence in the value. This can be used to decide which of two potentially
217 contradictory properties to believe. For example, the RAIT device examines the
218 source and surety of child devices' block sizes, prefering properties set by
219 the user (C<$PROPERTY_SOURCE_USER>) over others.
221 Set a property's source and surety with C<property_set_ex>:
222 $dev->property_set_ex("my_prop", 13, $PROPERTY_SURETY_BAD, $PROPERTY_SOURCE_DEFAULT);
223 The surety and source are returned after the property value in list context:
224 my ($val, $sur, $sou) = $dev->property_get("my_prop");
226 The available sureties are:
228 $PROPERTY_SURETY_GOOD
233 $PROPERTY_SOURCE_DEFAULT
234 $PROPERTY_SOURCE_DETECTED
235 $PROPERTY_SOURCE_USER
239 Some devices can perform more than one operation simultaneously, while others
240 are more limited. For example, a tape device is exclusive to a single process
241 while it is in use, while a VFS device can support concurrent reads and writes
244 As of this writing, device locking is not correctly implemented in many
245 devices; consult the source code and check with the Amanda developers before
246 depending on concurrent operation of devices.
250 When writing to a volume, an EOM (end-of-media) condition occurs when no more
251 space is available on the volume. Some devices (currently only those
252 supporting DirectTCP) distinguish a logical EOM (LEOM) from a physical EOM
253 (PEOM). The logical EOM comes some distance before the physical EOM, with
254 enough space left to finish a data block and write any additional bookkeeping
257 In such devices, the C<is_eom> attribute is set once LEOM is detected. Such
258 detection can happen in any method that writes to the volume, including
259 C<start>, C<start_file>, C<finish_file>, and C<finish>. API users that
260 understand LEOM should take this as a signal to complete writing to the device
261 and move on before hitting PEOM.
263 Devices which do not support EOM simply return a VOLUME_ERROR when the volume
264 is full. If this occurs during a C<write_block> operation, then the volume may
265 or may not contain the block - the situation is indeterminate.
267 =head2 Device Resources
269 Some device types have a "locking" mechanism that prevents other parts of the
270 system from accessing the underlying resource while an operation is in
271 progress. For example, a typical UNIX tape driver cannot be opened by two
274 Amanda Devices will lock the underlying resource when C<start> or C<read_label>
275 is called, and unlock the resource either when the Device object is
276 garbage-collected or in the C<finish> method. Thus in a calling sequence such as
284 the underlying resource remains locked for the entire sequence, even between
285 read_label and finish.
287 It is unwise to rely on Perl's garbage-collection to automatically release
288 resources. Instead, always explicitly release resources with a C<finish> call.
289 The Changer API is careful to do this in its C<release> method.
291 =head2 Member Variables
293 All member variables are implemented using accessor methods, rather than the
294 more common hashref technique. Use
296 print $dev->device_name, "\n";
300 print $dev->{'device_name'}, "\n";
302 The member variables are:
308 the current file number, if any
312 the current block number, if any
316 true if the device is in the middle of reading or writing a file
320 the name with which the device was constructed; note that this is not set until after open_device is finished -- it is an error to access this variable in an open_device implementation
324 the current access mode (C<$ACCESS_NULL>, or that supplied to start)
328 true if an EOF occurred while reading; also used by C<write_from_connection>
332 true if a write operation reached the end of the volume (end-of-medium)
334 =item C<volume_label>
336 the label of the current volume, set by start and read_label
340 the timestamp of the current volume, set by start and read_label
342 =item C<volume_header>
344 the header of the current volume, set by read_label
348 the device's error status (bit flags) as an integer
350 =item C<status_error>
352 the device's error status (bit flags) as a string
356 the device's error message
358 =item C<error_or_status>
360 the device's error message, if set, otherwise the same as C<status_error> --
361 use this to display error messages from devices
365 the device's currently configured block size. This is also available via the
366 BLOCK_SIZE property. Writers should use block_size-byte blocks, and readers
367 should initially use block_size, and expand buffers as directed by
370 =item C<min_block_size>
372 minimum allowed block size for this device
374 =item C<max_block_size>
376 maximum allowed block size for this device
380 =head2 Object Methods
382 =head3 configure($use_global_config)
386 Once you have a new device, you should configure it. This sets properties on
387 the device based on the user's configuation. If C<$use_global_config> is true,
388 then any global C<device_property> parameters are processed, along with
389 tapetype and other relevant parameters. Otherwise, only parameters from the
390 device definition (if the device was opened via an alias) are processed.
392 This method is I<deprecated>. All access to Devices should be via the Changer
393 API (see L<Amanda::Changer>), which implements its own, more advanced method of
394 configuring devices. The C<configure> method may be removed in a future
399 $status = $dev->read_label();
401 This function reads the tape header of the current volume, returning the
402 Device's status (see "Error Handling", above). Since this is often the first
403 function to accses the underlying hardware, its error status is the one most
404 often reported to the user. In fact, C<amdevcheck(8)> is little more than a
405 wrapper around read_label.
407 The method sets the following member variables:
411 =item C<volume_header>
413 if any header data was read from the volume, it is represented here. The
414 header's type may be F_WEIRD if the header was not recognized by Amanda.
416 =item C<volume_label>
418 if read_label read the header successfully, then volume_label contains the
423 smililarly, if read_label read the header successfully, then volume_time
424 contains the timestamp from the header
430 $succss = $dev->start($ACCESS_WRITE, $label, $timestamp);
432 Start starts the device and prepares it for the use described by its second
433 parameter. This function can be called regardless of whether C<read_label> has
436 If the access mode is C<$ACCESS_WRITE>, then the label and timestamp must be
437 supplied (although leaving the timestamp undef will use the current time), and
438 they will be used to write a new volume header. Otherwise, these parameters
441 On completion, start leaves the device's C<access_mode>, C<volume_label> and
442 C<volume_time> member variables set, by reading the tape header if necessary.
443 Note that in mode C<$ACCESS_APPEND>, the C<file> member variable is not set
444 until after C<start_file> has been called.
448 $success = $dev->start_file($header);
450 This method prepares the device to write data into a file, beginning by writing
451 the supplied header to the volume. On successful completion, the device's
452 C<file> is set to the current file number, C<block> is zero, and C<in_file> is
453 true. If the volume is out of space, the C<is_eom> member is set to true and
454 the method returns false with status C<DEVICE_STATUS_VOLUME_ERROR>.
458 # (not available from Perl)
459 success = device_write_block(dev, blocksize, buf);
461 This method writes a single block of data to the volume. It is only available
462 from C -- Perl code should not be handling raw data, as it is far too slow.
463 Use the transfer architecture (L<Amanda::Xfer>) for that purpose.
465 The C<blocksize> must be the device's block size, unless
466 this is a short write. A short write must be the last block
467 of a file. Some devices will zero-pad a short write to a full
468 blocksize. This method returns false on error. If the volume is
469 out of space, C<is_eom> is set and the method returns false with
470 status C<DEVICE_STATUS_VOLUME_ERROR>. Note that not all devices can
471 differentiate an EOM condition from other errors; these devices will
472 set C<is_eom> whenever the situation is ambiguous.
474 This function ensures that C<block> is correct on exit. Even in an
475 error condition, it does not finish the current file for the caller.
477 =head3 write_from_fd (DEPRECATED)
479 my $qfd = Amanda::Device::queue_fd_t->new(fileno($fh));
480 if (!$dev->write_from_fd($fd)) {
481 print STDERR $qfd->{errmsg}, "\n" if ($qfd->{errmsg});
482 print STDERR $dev->status_or_error(), "\n" if ($dev->status());
485 This method reads from the given file descriptor until EOF, writing the data to
486 a Device file which should already be started, and not returning until the
487 operation is complete. The file is not automatically finished. This method is
488 deprecated; the better solution is to use the transfer architecture
491 This is a virtual method, but the default implementation in the Device class
492 uses C<write_block>, so there is no need for subclasses to override it.
496 $success = $dev->finish_file();
498 Once an entire file has been written, finish_file performs any
499 cleanup required on the volume, such as writing filemarks. On exit,
500 C<in_file> is false. If the device runs out of space while finishing
501 (e.g., the filemark does not fit), then this method returns false
502 with status C<DEVICE_STATUS_VOLUME_ERROR> and C<is_eom> is set.
504 This function should not be used while reading -- instead, just seek
509 $header = $dev->seek_file($fileno);
511 In C<$ACCESS_READ>, C<seek_file> sets up the device to read from file
512 C<$fileno>. This function is not available in C<$ACCESS_WRITE> and
513 C<$ACCESS_APPEND>. It returns the header from the requested file on success, or
516 If the requested file doesn't exist, as might happen when a volume has had
517 files recycled, then C<seek_file> will seek to the next file that does exist. The
518 file this function selected is indicated by the C<file> member variable on exit.
519 If the requested file number is exactly one more than the last valid file, this
520 function returns a C<$F_TAPEEND> header.
522 As an example, on a volume with only files 1 and 3:
524 $dev->seek_file(1) returns header for file 1, $dev->file == 1
525 $dev->seek_file(2) returns header for file 3, $dev->file == 3
526 $dev->seek_file(3) returns header for file 3, $dev->file == 3
527 $dev->seek_file(4) returns a tapend header, $dev->file == 4
528 $dev->seek_file(5) returns NULL/undef
530 On exit, C<is_eof> is false, C<in_file> is true unless no file was found (tapeend or NULL), C<file> is the discovered file, and C<block> is zero.
534 $success = $dev->seek_block($block);
536 After seeking to a file, the caller can optionally seek to a particular block
537 in the file. This function will set C<block> appropriately. Note that it may
538 not be possible to detect EOF, so this function may fail to set C<is_eof> even
539 though a subsequent C<read_block> will return no data.
543 # (not available from Perl)
544 bytes_read = device_read_block(dev, buffer, *blocksize);
546 This method is the complement of C<write_block>, and reads the next block from
547 the device, or returns -1 on error. Pass a buffer and its size. If the buffer
548 is not big enough, no read is performed, the parameter C<blocksize> is set to
549 the required blocksize, and the method returns 0. As a special case, passing a
550 C<NULL> buffer and C<*blocksize == 0> is treated as a request for the required block
551 size. It is not an error to pass a buffer that is too large (and, in fact, this
552 is precisely the effect of setting the C<read_block_size> configuration
555 On EOF, this method returns -1, but sets C<is_eof> and leaves the device's
556 status set to C<$DEVICE_STATUS_SUCCESS>. Some devices may be able to detect EOF
557 while reading the last block, and will set C<is_eof> at that time. Others must
558 wait for the next read to fail. It is never an error to call C<read_block>
559 after an EOF, so there is no need to check C<is_eof> except when C<read_block>
562 =head3 read_to_fd (DEPRECATED)
564 queue_fd_t *qfd = queue_fd_new(fd, NULL);
565 my $qfd = Amanda::Device::queue_fd_t->new(fileno($fh));
566 if (!$dev->read_to_fd($fd)) {
567 print STDERR $qfd->{errmsg}, "\n" if ($qfd->{errmsg});
568 print STDERR $dev->status_or_error(), "\n" if ($dev->status());
571 This method reads the current file from the device and writes to the given file
572 descriptor, not returning until the operation is complete. This method is
573 deprecated; new uses of devices use the transfer architecture
576 This is a virtual method, but the default implementation in the Device class
577 uses C<read_block>, so there is no need for subclasses to override it.
581 $success = $dev->finish();
583 This undoes the effects of start, returning the device to a neutral state
584 (C<$ACCESS_NULL>). It will also release any resources acquired by
585 C<read_label>, even if C<start> was not called. After C<finish>, it is not an
586 error to call C<start> again, even with a different mode.
590 $success = $dev->recycle_file(fileno);
592 On devices that support it, this removes the indicated file from the volume,
593 presumably freeing its space to be used for other files. File numbers of
594 existing files will not change, so this operation may leave "holes" in the
595 sequence of file numbers. See C<seek_file> to see how this is handled.
597 This method cannot be called while in a file, nor while in C<$ACCESS_READ>
602 $success = $dev->erase(fileno);
604 On devices that support it, this erases all data from the volume, presumably
605 freeing the space. This method must be called before start and after finish --
606 that is, while the device is in a neutral state (C<$ACCESS_NULL>). You can
607 detect whether or not this operation is supported using the C<full_deletion>
612 $success = $dev->eject();
614 On devices that support it, this eject the volume. This method can be called
615 before start and after finish.
617 =head3 directtcp_supported
619 $supp = $dev->directtcp_supported();
621 This method returns TRUE if the DirectTCP-related methods (C<listen>,
622 C<accept>, C<write_from_connection>, and C<read_to_connection>) are implemented
627 $addrs = $dev->listen($for_writing);
629 The C<listen> method starts the device listening for an incoming DirectTCP
630 connection. The method returns a set of IP:PORT pairs to which a TCP
631 connection can be made. The boolean C<for_writing> is TRUE if
632 this connection will be used to write to the device.
634 This method can be called at any time, but between the time C<listen> is called
635 and when C<accept> returns, no other methods of the device should be called.
637 The return value might look like:
639 $addrs = [ [ "127.0.0.1", 9382 ] ]
641 In C, the memory for these addresses remains the responsibility of the device,
642 and will remain unchanged until C<accept> returns.
646 $conn = $dev->accept();
648 This method accepts a connection to one of the addresses returned by C<listen>,
649 returning an established DirectTCPConnection object (see below). It returns
650 C<undef> on failure. Note that this method may block indefinitely if no
651 connection ever occurs. The C implementation returns an already-referenced
652 connection object, so the caller should call C<g_object_unref> when the
653 connection is no longer needed.
655 =head3 use_connection
657 my $ok = $dev->use_connection($conn);
659 Call this method to use a DirectTCPConnection object created with another
660 device. The method must be called before the device is started (so
661 C<access_mode> is C<$ACCESS_NULL>), as some devices cannot support switching
662 connections without rewinding. Any subsequent C<read_from_connection> or
663 C<write_to_connection> calls will use this connection.
665 =head3 write_from_connection
667 ($ok, $actual_size) = $dev->write_from_connection($size);
669 This method reads data from the DirectTCPConnection specified with
670 C<use_connection> or returned from C<accept> and writes it to the volume. It
671 writes at most C<$size> bytes, and returns the number of bytes written in
672 C<$actual_size>. On error, C<$ok> is false.
674 When an EOF is received over the connection, signalling the end of the data
675 stream, then this method returns without error (C<$ok> is true), with
676 C<$actual_size> indicating the number of bytes written to the device (which may
677 be zero). In this case, the C<is_eof> attribute is true on return.
679 Similarly, when the device encounters logical EOM in this method, it returns
680 the total bytes transferred in C<$actual_size>, with C<$ok> true, and the
681 C<is_eom> attribute true. No data is lost. If writes continue until physical
682 EOM, data may be lost.
684 =head3 read_to_connection
686 ($ok, $actual_size) = $dev->read_to_connection($size);
688 This method is similar to C<write_from_connection> but the data flows in the
689 opposite direction. It reads at most C<$size> bytes, and returns the total
690 number of bytes read in C<$actual_size>.
692 When the method encounters an EOF, it stops early and returns successfully with
693 the number of bytes actually read (which may be zero).
697 Get a property value, where the property is specified by name. See "Properties", above.
701 Set a simple property value. See "Properties", above.
703 =head3 property_set_ex
705 Set a property value with surety and source. See "Properties", above.
709 This module defines a large number of constant scalars. These constants are
710 available from the package namespace (e.g., C<$Amanda::Device::ACCESS_WRITE>),
711 or imported with the C<:constant> import tag.
713 =head2 DirectTCPConnection objects
715 The C<accept> method returns an object to represent the ongoing DirectTCP
716 connection. This object is mostly useful as a "token" to be passed to
717 C<write_from_connection> and C<read_to_connection>. In particular, a
718 connection created by one device can be used with another device; this is how
719 DirectTCP dumps are spanned over multiple volumes.
721 The class does have one critical method, though:
725 This method closes the connection, releasing all resources allocated to it. It
726 can be called at any time, whether the remote side has closed the connection