Database Shadows - Firebird

Firebird has a capability that can enable immediate recovery of a database in case of disk failure, network failure, or accidental filesystem deletion of the database. Called shadowing, it is an internal process that maintains a real-time, physical copy of a database. Whenever changes are written to the database, the shadow receives the same changes simultaneously.

An active shadow always reflects the current state of the database. However, although shadowing has obvious benefits as a hedge against hardware failure, it is not an online replication system.

Benefits and Limitations of Shadowing

The main benefit of shadowing is that it provides a quick way to recover from a hardware disaster. Activating a shadow makes it available immediately. It runs invisibly to users as an extra loop in the data-writing process, with minimal attention from the DBA.

Creating a shadow does not require exclusive access to the database, and the shadow can be maintained in one or many files, in controlled spaces of the server’s storage system.

However, shadowing is not a protection against corruption. Data written to the shadow is an exact copy of what is written to the database, warts and all. If user errors, intermittent disk faults, or software failures have corrupted data, then the same corrupt data will show up in the shadow.

Shadowing is an all-or-nothing recovery method. It has no provision to recover some pieces and ignore others or to revert to a specific point in time. It must live in the same filesystem as the server; a shadow must be written to fixed hard drives local to the server. It cannot be written to shares, non-local filesystems, or removable media.

Important Caveats

Shadowing is not a substitute for backup. Do not be lulled into the belief that shadowing is in any way a substitute for regular backup and periodic restores.

  • A shadow file is no less vulnerable to the “slings and arrows of outrageous fortune” than is any other file in your system.
  • One lost or damaged shadow file makes the whole shadow useless.
  • A dying disk or a flaky memory module is capable of great mischief before it brings your database down totally. Every piece of mischief will be faithfully written to the shadow.

In addition, a shadow cannot accept connections. Never try to connect to a shadow or interfere with it using system or database tools. The server “knows” what it has to do to a shadow to convert it to an active database.

Implementing Shadowing

Firebird has DDL syntax for creating and dropping shadows with various optional clauses to specify location, operating mode, and file sizes. Altering a shadow requires dropping the existing one and creating a new one with new specifications.

A shadow that is a single disk file is referred to as a shadow file. A multiple-file shadow—which can be distributed across more than one disk—is known as a shadow set. Shadow sets are grouped by a shadow set number.

Shadow File Location and Distribution

A shadow should be created in different fixed disk spaces from the location or locations of the active database files, since one of the main purposes of shadowing is to provide fallback in case of disk failure.

Disk space for shadowing must be physically attached to the Firebird server’s host machine. Files in a shadow set can be distributed across multiple disks to improve file I/O and space allocation. Like file specifications for databases, those for shadows are platform specific.

Shadow Options

Mode (AUTO or MANUAL)

Mode settings, automatic (with or without the conditional attribute) or manual, determine what occurs if something happens to make the shadow unavailable.

AUTO mode is the default. It allows the database to keep running in the event that the shadow becomes inoperable.

  • There will be a window of time during which no shadowing happens and the DBA may be unaware of the loss.
  • If the lost shadow was created with the CONDITIONAL attribute, Firebird creates a new shadow automatically, if it can.
  • If shadowing is not conditional, it will be necessary to re-create a shadow manually.

MANUAL mode prevents further access to the database in the event that the shadow becomes unavailable. Choose it if continuous shadowing is more important than continuous operation of the database.

To allow connections to resume, the administrator must remove the old shadow file, delete references to it, and create a new shadow.

Conditional Shadowing

One of the ways a shadow could become unavailable is by taking over as the main database in the event of the hardware “death” of the existing database—that is, after all, the whole idea behind shadowing!

Single-File vs. Multi-File Shadows

By default, a shadow is created as a “set” consisting of one file. However, a shadow set can comprise multiple files. As a database—and hence its shadow—grow in size, the shadow can be respecified and regenerated with more files to accommodate the increased space requirements.

Creating a Shadow

A shadow can be created without needing exclusive access or affecting any connected users. The DDL statement CREATE SHADOW creates a shadow for the database to which you are currently connected.

This is the syntax:

Single-File Shadow

Let’s assume we are on a Linux server, logged into the database employee.gdb, which is located in the examples directory beneath the Firebird root. We have decided to shadow the database to a partition named /shadows. To create a single-file shadow for it there, we use the statement

CREATE SHADOW 1 '/shadows/employee.shd';

The shadow set number can be any integer. A page size is not included, since the PAGE_SIZE attribute is taken from the specification of the database itself.

Use the isql command SHOW DATABASE to check that the shadow now exists:

Multi-File Shadow

The syntax for creating a multi-file shadow is quite similar to that for creating a multifile database: the secondary shadow file specifications are “chained” to the primary file specification, with file specs and size limits for each file.

In this example, assume we are logging into employee.gdb in its default Win32 location. We are going to create a three-file shadow on drives F, H, and J, which are partitions on hard drives in the server’s filesystem.

File sizes allocated for secondary shadow files need not correspond to those of the database’s secondary files.

The primary file, employee1.shd, is 10,000 database pages in length, and the first secondary file, employee2.shd, is 20,000 database pages long. As with the database, the shadow’s final secondary file, employee3.shd, will grow as needed until space is exhausted on partition J or the filesystem limit is reached.

Alternatively, we can specify starting pages for the secondary files instead of absolute lengths for the primary and non-final secondary files:

You can verify in isql:

Manual Mode

The preceding examples created shadows in the default AUTO mode. Now, suppose we have a requirement that work in the database must be stopped at all times when shadowing is disabled for any reason. In this case, we want to stipulate that the shadow is created in MANUAL mode (refer to notes earlier in this topic). In order to tell the Firebird engine that we want this rule enforced, we create the shadow using the reserved word MANUAL in the CREATE SHADOW statement:

CREATE SHADOW 9 MANUAL '/shadows/employee.shd';

Now, if the shadow “kicks in” because we lose the main database file, or if the shadow becomes unavailable for some other reason, the administrator must drop the old shadow file and create a new shadow before clients can resume connecting.

A Conditional Shadow

In both of the preceding examples, the CREATE SHADOW specifications leave the database unshadowed after the shadow becomes unavailable, whether by becoming detached from the database or by its having “kicked in” as the active database upon the physical death of the original database.

In the case of a MANUAL mode shadow, that is the situation we want. We choose this mode because we want database connections to stay blocked until we manually get a new shadow going.

In the case of an AUTO mode shadow, database connections can resume after the death, as soon as the shadow kicks in. No shadowing will occur from this point until the admin manually creates a new shadow. Also, if a shadow becomes detached for some reason, the admin will not know about it. Either way, we have a window where shadowing has lapsed.

As described earlier in this topic, we can improve this situation by including the CONDITIONAL keyword in the AUTO mode shadow’s definition. The effect will be that, when the old shadow becomes unavailable, the Firebird engine will do the necessary housekeeping and create a new shadow automatically, for example:

SHADOW 33 CONDITIONAL '/shadows/employee.shd';

Increasing the Size of a Shadow

At some point, it may become necessary to increase the sizes or numbers of files in a shadow. Simply drop the shadow (as described in the next section) and create a new one.

Dropping a Shadow

A shadow needs to be dropped in the following situations:

  • It is a manual shadow that has been detached from the system for some reason. Dropping the compromised shadow is a necessary precursor to creating a new shadow and resuming database service.
  • It is an unconditional automatic shadow that has been offline because of some system event. It needs to be regenerated to restore its integrity.
  • You need to resize the files in a shadow, add more files, or set up a new shadow with different attributes.
  • Shadowing is no longer required. Dropping the shadow is the means to disable shadowing.

Dropping a shadow removes not just the physical files but also the references to it in the metadata of the database. To be eligible to run the command, you must be logged into the database as the user who created the shadow, the SYSDBA, or (on POSIX) a user with root privileges.

Syntax

Use this syntax pattern for DROP SHADOW:

DROP SHADOW shadow_set_number;

The shadow set number is a compulsory argument to the DROP SHADOW command. To discover the number, use the isql SHOW DATABASE command while logged into the database.

The following example drops all of the files associated with the shadow set number 25:

DROP SHADOW 25;

Using gfix –kill

The command-line gfix housekeeping utility tool has a –kill switch that conveniently submits a DROP SHADOW command internally to clean up an unavailable shadow. After executing this command, it will be possible to proceed with creating a new shadow.

For example, to drop the unavailable shadow on our employee database on POSIX, type

On Win32, type


All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd DMCA.com Protection Status

Firebird Topics