In normal or high redundancy, ASM creates and maintains two or more copies of the ASM extent: a primary mirrored extent andsecondary/tertiary mirrored extent.In Oracle Database 10g,the ASM instance always reads the primary copy of the mirrored extent unless the primary cannot be accessed, and then the secondary will be read.
Oracle Database 11g introduces the ASM preferred mirror read, which provides the database with the ability to read from the secondary mirrored extent first before reading the primary mirrored extent.
This feature provides the greatest benefit for extended stretched RAC cluster implementations. With stretched RAC clusters, diskgroups are configured with the local primary copy in the local data center and the failure diskgroups at the remote data center.
The remote RAC instance is now able to read a local copy of the ASM extent, thus significantly reducing the amount of network latency to read from the remote data center and effectively increasing performance of the appli cations.If that localized read fails, then ASM will make an effort to read from the secondary mirrored extent from the remote failure group.
In Oracle Database 10g, the ASM instance always reads the primary copy of the mirrored extent, even if this means accessing the extent across the interconnect,in other words,remote extent access.If the interconnect latency is high, then any remote node access will impact database performance.
Preferred Read Setup
Oracle enables ASM preferred reads through the initialization parameterasm_ preferred_read_failure_groups.This parameter specifies the preferred read mirror names and allows each ASM instance to read from the localized mirror copy.
This parameter accepts values in pairs of diskgroup_name.failure group name delimited by a period.This parameter will guarantee localized reads for the ASM instance.You can provide multiple values to this parameter by putting commas between the values.
This parameter is dynamically adjustable and can be modified with the alter system command. Even though this parameter is dynamically adjustable, this parameter does not take effect until the diskgroup is remounted.
The alter system command updates the SGA structure of a disk in RDBMS to reflect whether it is a preferred read disk.For normal and high redundancy diskgroups,there should be one failure group on each site of the extended cluster.For normal redundancy diskgroups, there should be two sites. Likewise,for high redundancy diskgroups, the extended cluster configurations should be three sites.If there are more diskgroups in one site over another (in other words,four failure groups and three extended RAC clusters),extents will end up mirrored to the same site,thus eliminating the high-availability benefit of setting up high redundancy.
Here’s the syntax to create a normal redundancy so you can see how it is applicable to the preferred mirror reads:create diskgroup DATA normal redundancy
The fg1 failgroup is recognized as a localized read from RAC node #1 in SiteA, while it is recognized as a remote mirror read from RAC node #2 in SiteB.Likewise, fg2 failgroup is recognized as a localized mirror read from node #2 in SiteB and as a remote mirror read from RAC node #1 in SiteA.For the SiteA servers,the ASM instances should read from the SAN storage devices that are from the SiteA data center:asm_preferred_read_failure_groups = data.fg1
Set the parameter for the database servers in SiteB to the following:asm_preferred_read_failure_groups = data.fg2
You can also configure the preferred read failure groups using Enterprise Manager Database Console.This option is the Preferred Read Failure Groups field on the Configuration tab of the ASM home page. You can specify a list of failure groups delimited by commas whose member disks will serve as the preferred read disks for this node.
Figure ASM diskgroup preferred read
Monitoring Preferred Reads
Oracle Database 11g adds the new column PREFERRED_READ to the V$ASM_DISK dynamic view.This column holds a Y or an N value to designate that the read is a localized and preferred read failure group. You can also view the performance characteristics of the ASM preferred read failure group by querying V$ASM_DISK_IOSTAT. The result of this view is to provide information at the ASM instance level:SQL> select instname, failgroup,
ASM Restricted Mode
Oracle Database 11g ASM implements the restricted start-up option for maintenance mode.While in restricted mode,only the starting ASM instance has exclusive access to the disk groups.When in restricted mode, databases will not be permitted to access the ASM instance.You can see that startup restrict; will mount all the diskgroups in restricted mode:SQL> startup restrict;
When you try to start a database while the ASM instance is in restricted mode, you will receive the following error message:SQL> startup
The restricted option can be specified at the diskgroup level to provide a granular level of restriction.
Let’s look at how to enable restricted access at the diskgroup level:SQL> alter diskgroup data mount restricted;
The restricted option can be used in RAC or non-RAC environments to perform a maintenance task such as a rebalance operation. In restricted mode, a re balance mitigates ASM to ASM extent relocation messaging, making the rebalance operation much faster.
ASM Diskgroup Maintenance
Oracle Database 11g introduces the force option to the mount and drop diskgroup syntax.In Oracle Database 10g, to mount a diskgroup, DBAs would issue the following command:SQL> alter diskgroup data mount;
In Oracle Database 11g, you can force the mount of the diskgroup even if not all the disks participating in the diskgroup are available:SQL> alter diskgroup data mount force;
By default, the mount of the diskgroup is done with the noforce option. All disks must be available for the mount to succeed.With the force option, ASM simply offlines any disks that are missing and mounts the diskgroup with the available disks.
This option can cause problems if used incorrectly.The issue could be something as simple as an incorrect asm_diskstring parameter.By using the force option to mount a diskgroup, you can cause more problems than not.
The force option should be used cautiously.The force option can be leveraged with ASM redundancy diskgroups.ASM will force a mount of a diskgroup even with missing or damaged failure groups. In a RAC environment, only one RAC node may mount a diskgroup with the force option. Once a diskgroup is mounted, the remaining nodes will receive an alert when it tries to mount the diskgroup with the force option.
In the world of ASM, it is common practice to use the dd utility to clear out the header of disks.In Oracle Database 10g, in order to drop a diskgroup, it had to be mounted.If the diskgroup cannot be mounted, you can clear the disk(s) using the dd command.
In Oracle Database 11g, ASM introduces the force option to the drop disk group command.The force option can be used if the target diskgroup cannot be mounted.The force option marks the headers of disks with a “FORMER” status.
In Oracle Database 11g, Oracle lifts the requirement that the diskgroup has to be mounted to drop the diskgroup.It is considered a best practice to mount the diskgroup to issue the drop dis kgroup command.Only when the diskgroup cannot be mounted should you use the force option.You can use the following syntax to drop the diskgroup:SQL> drop diskgroup data force;
When using the force option, you can include the including contents option:SQL> drop diskgroup data force including contents;
Similarly,you can perform a drop diskgroup operation using Enterprise Manager Database Console. From the Diskgroup main page, go to the ASM Instance Main page and then the Diskgroup tab, and you will see a Remove button. You can drop a particular diskgroup by selecting a diskgroup from the list and clicking the Remove button. On this screen,you have the option to drop with force or with out force, as displayed in Figure
Figure ASM diskgroup drop
You will also notice that there is a drop-down list for the rebalance operation. You can choose your power limit for the rebalance after the drop operation.ASM performs some minor checks prior to dropping a diskgroup.First, ASM checks to see whether the diskgroup is being used by another ASM instance in the same SAN storage subsystem.
If so, ASM checks to see whether the ASM diskgroup is participating in the same cluster. If so,the command will return an error. Next, ASM will check to see whether the diskgroup is mounted by any other cluster.If so, the command will return an error.Please use extreme caution when dropping diskgroups with the force option.
The effects are irreversible, and the damage can be severe.ASM does perform minor checks, but they are not definitive. After a successful drop force of a diskgroup, you can view the ASM alert logs for details:SQL> alter diskgroup data dismount
You can also perform ASM diskgroup mount and dismount operations using Enterprise Manager Database Console.The Diskgroup Mount/Dismount button is located on the Diskgrou main screen on the ASM Home ➤ Disk Groups tab. When you choose to mount a particular diskgroup by selecting a disk group from the list and clicking the Mount button, you will be directed to the Diskgroup Mount page, as shown in Figure
Figure ASM diskgroup mount
You will also notice that you can check a box to mount the diskgroup in restricted mode.On the other hand, you may have to dismount a diskgroup. You can dismount a particular diskgroup by selecting a diskgroup from the list and clicking the Dismount button. On this screen,you have the option to dismount with force or without force, as displayed in Figure
Figure ASM diskgroup dismount
Starting in Oracle Database 11g, you can validate the internal consistency of ASM diskgroup metadata using the alter diskgroup ...check command. The check clause does the following:
The alter diskgroup check command is applicable to a specific file in a diskgroup, one or more disks in a diskgroup, or specific failure groups in a diskgroup.This command also checks all the metadata directories.The following example checks the DATA diskgroup:SQL> alter diskgroup data check;
A summary of errors is logged in the ASM alert log file. Here’s an excerpt from the alert log file:SQL> alter diskgroup data check
The default behavior of the check clause is to repair the discovered errors. However,you can choose the norepair option if you do not want ASM to resolve the errors automatically.The good news is that you will still receive alerts about inconsistencies:SQL> alter diskgroup data check norepair;
In the previous release, you had the following options to the check
command:all,disk,disks in failgroup, and file.These options are now deprecated.If these options are specified,the commands will continue to work, but messages will be generated to the alert log.
You can also perform diskgroup checks using Enterprise Manager Database Console.The diskgroup Check button is located on the Diskgroup tab of the ASM Home page, as shown in Figure.
Figure ASM diskgroup Check button
Clicking the Check button will direct you to the Check Diskgroup page.On this screen,you have the option to check the diskgroup with or without the repair option,as displayed in Figure.
Figure ASM diskgroup check
Oracle Database 11g introduces a new concept called ASM attributes.Attributes provide a granular level of control for DBAs at the diskgroup level. Here are the attributes you can set:
The attributes for the diskgroup can be established at create diskgroup time or can be modified using the alter diskgroup command later.All of the diskgroup attributes can be queried from the V$ASM_ATTRIBUTE view.
Now we’ll show how attributes can be set and modified for ASM diskgroups.First, we’ll create a diskgroup with 10.1 diskgroup compatibility and then advance it to 11.1 using the alter diskgroup command.You can use the compatible.asm attribute to advance this attribute to 11.1:create diskgroup data
Alternatively, if you use ASMLIB, you can create the diskgroup using the ASMLIB disk names:create diskgroup data
Now, we’ll show the syntax to advance the disk group ASM attribute to 11.1. Please remember that once you advance an attribute to a higher version,you cannot reverse this action.Once you set any attributes to 11.1, you cannot go back to 10.x.SQL> alter diskgroup data set attribute'compatible.asm' = '220.127.116.11.0';
You can also set ASM and RDBMS attributes for the diskgroup using Enterprise Manager Database Console.The Advanced Attributes section of the diskgroup is on the Diskgroup main screen on the ASM Home ➤ Disk Groups tab. Click the diskgroup, as shown in Figure.
Figure ASM diskgroup attributes Edit button
Figure ASM diskgroup attributes
On the successful advancement of the diskgroup to11.1, the following message is listed in the ASM alert log file:NOTE: Advancing ASM compatibility to 18.104.22.168.0 for grp 1
Let’s query the V$ASM_ATTRIBUTE view to confirm that the compatible is truly set:&select name, value
Using the DATA diskgroup we created earlier,let’s change the compatible.rdbms attribute to 11.1:SQL> alter diskgroup data set attribute'compatible.rdbms' = '11.1';
By querying V$ASM_ATTRIBUTE, you can see that the compatibility for the RDBMS is set:select name, value
You can specify a combination of attributes at diskgroup creation time. We’ll show another example where the au_size and compatible.asm attributes are specified in a single create diskgroup command:create diskgroup fra disk '/dev/raw/raw11',
Allocation Unit (AU) Sizes
Prior to Oracle Database 11g, the AU size could not be specified at diskgroup creation time.All AUs were 1MB.There is a workaround, but not very many DBAs are aware that you can have multiple au_sizes in Oracle Database 10g.Specifying au_size in Oracle Database 10g means setting an underscore initialization parameter and creating the diskgroup.
If you want multiple AU sizes, you must set the initialization parameter and bounce the ASM instance with the new AU size.The following underscore initialization parameters allow a 16MB AU size and 1MB AU stripe size.This is recommended only for VLDB databases or may be suitable for databases that have large objects (BLOBs and CLOBs).
In Oracle Database 11g,the au_size attribute can be specified only at diskgroup creation time. Because it involves storage characteristics, this attribute cannot be modified using the alter diskgroup command.
Starting in Oracle Database 11g,you can set the ASM allocation unit (AU) size from 1MB all the way to 64MB in powers of 2.As of Oracle Database 11g,the valid extent sizes are 1, 2, 4, 8, 16, 32, or 64MB.
The larger AUs can be beneficial for large VLDB databases or data warehouses that perform large sequential reads.In addition, organizations that store BLOB or SecureFiles inside the database can benefit from larger AUs.
Variable-size extents provide support for larger ASM files.Moreover,this feature reduces the SGA requirements to manage the extent maps in the RDBMS instance.Setting the AU to a higher number also reduces the metadata space usage since it reduces the number of extent pointers associated with the metadata.
Moreover,variable-size extents can significantly improve database open time and reduce memory utilization in the shared pool.Variable-size extents allow you to support databases that are hundreds of TB and even several PB in size.
Variable-size extents dynamically change extent size depending on how many AUs have been allocated.The management of variable-size extents is automatic and does not require manual intervention.
The ASM variable extent feature will kick in and start allocating 8× AU extent sizes after 20,000 extents have been allocated.Variable-size extents are similar to the uniform extent allocation in the database and are allocated in 1, 8,and 64 AU chunks.
This is automatically handled by ASM. Variable-size extents minimize the overhead associated with maintaining ASM metadata. After 40,000 extents, the extents are allocated at 64× AU size. The largest size extent supported in Oracle Database 11g will be 64MB.
ASM files larger than 20GB and up to 128TB are great candidates for variable-size extents.
We’ll now take the default 1MB allocation unit and provide an example of how this works.When the file hits 20GB (20,000 extents),the extent size will change from 1MB to 8MB since 8MB is 8 times the AU.At this point, AU will grab 8MB chunks until it reaches 40,000 extents; 20,000 additional extents at 8MB will be 160,000GB.When the file reaches the 180GB threshold (160GB + 20GB), the AU will change to 64MB.
There is a rare incident where large amounts of noncontiguous small data extents are allocated and freed and can cause situations when large contiguous space becomes unavailable.This causes fragmen tation to occur in the diskgroup.
The remedy is simple—rebalance the diskgroup to reclaim large contiguous space.If you do not perform a rebalance, it can cause a slight degradation in performance since ASM automatically performs defragmentation if extent allocation cannot sequester the storage.
Manually Allocate Larger Extents
Starting with Oracle Database 11g,you can allocate larger size extents when you create a disk group.Instead of the 1MB-sized extent,you can create diskgroups with larger AU.You can change the allocation unit size to 16MB for the DATA diskgroup using the create diskgroup command and setting the appropriate attributes for the disk group,as displayed here:SQL> create diskgroup data
RDBMS and ASM Compatibility
Two kinds of compatibility settings are relative to ASM and the database. When DBAs think about compatibility,they think about the initialization para meter in the init.ora file or spfile that dictates which version of function ality will be available to the ASM or database instance.
The compatible parameter is one of the two kinds of compatibility settings.The compatible parameter can be set for either ASM or the database instance. Here are the valid compatible values for both the ASM and database instance:
Obviously,10.1 is the lowest level of compatibility since ASM was introduced as a feature in 10.1.Setting the initialization parameter to a lesser value than the software release will exclude availability of the new features introduced in the new release.
For example,if the compatible para meter is set to 10.2 for an 11.1 ASM instance,the new Oracle Database 11g features will not be available such as drop diskgroup with the force option as mentioned earlier.Similarly, the compatible parameter for the database plays a key role in negotiating what features are supported.
The other compatibility setting applies to ASM diskgroups and the functionality provided at the diskgroup level.There are attribute settings at the diskgroup level that control what features are available to the ASM diskgroup and which capabilities are available at the database level.
These attributes are called ASM compatibility (compatible.asm) and RDBMS compatibility (compatible.rdbms).At each diskgroup level,you can adjust these two compatibility settings to meet the business or technology requirements.The disk group compatibility information is stored in the diskgroup metadata and provides the multiple-level support of different versions (10g, 10g Release 2, and 11g) of the databases.
The terms RDBMS compatibility and ASM compatibility refer exclusively to diskgroup-level compatibility.RDBMS compatibility is classified by the compatible.rdbms attribute. Surprisingly,the compatible.rdbms attribute defaults to 10.1 in Oracle Database 11gThis attribute dictates the minimum compatible version setting of a database that is allowed to mount the disk group,and controls the format of the messages exchanged between the ASM and RDBMS instances.
ASM diskgroup compatibility is defined by the compatible.asm attribute and controls the persistent format of the disk ASM metadata structures. Similar to the compatible.rdbms attribute, ASM defaults the compatible.asm attribute to 10.1.The general rule to remember is that the compatible.asm attribute must always be greater than or equal to the RDBMS compatibility level.
The compatible attributes can be either in short form or in long form for the version number. The range of values for compatible.rdbms or compatible.asm can be from one version all the way to five versions: 11.1 to 22.214.171.124.0. For example,the compatible attribute can specified as 11.1 or 126.96.36.199. Similarly, for Oracle Database 10g environments, it can be set to 10.1 or 10.1.0.0.0.
The combination of compatible.asm and compatible.rdbms attributes control the persistent format of the disk ASM metadata structures.These attributes influence whether a database instance can mount a diskgroup.Three things determine whether a database instance is allowed to mount a disk group:
Even though you upgrade an ASM instance to Oracle Database 11g,the compatible.asm and compatible.rdbms attributes for the diskgroups still remain at 10.1,which is the default and the lowest attribute level for ASM.You can see here that the ASM compatibility and RDBMS compatibility are still at 10.1 for the FRA diskgroup:select name, block_size,
ASM instances can support multiple databases on different versions.Each of the databases can have a different compatibility setting.The key point to remember is that the compatible initiali zation parameter must be greater than or equal to the RDBMS compatibility of all diskgroups used by the database.
As stated earlier, ASM compatibility of a diskgroup can be set to 11.0, while its RDBMS compatibility could be 10.1.This implies that the diskgroup can be managed only by ASM software version 11.0 or higher, while any database software version must be 10.1 or higher.To determine the software version and compatibility setting of the database, you can query the V$ASM_CLIENT view, as displayed hereselect db_name,
Fast Mirror Resync
In Oracle Database 10g,ASM offlines a disk when it is not able to write an extent or access the disk and is not needed anymore,shortly after the disk is dropped from the diskgroup.At this point,ASM will perform a rebalance on the extents of the surviving disk members in the diskgroup using the mirror extent copies.
This rebalance operation is extremely costly and can take hours. Even for nondisk problems such as bad cables or problems with HBA or controllers, disks may get dropped from the diskgroup, and rebalance activity may occur.ASM is ignorant of what is causing the issue. ASM just knows that it is not able to complete a write operation.
In Oracle Database 11g,Oracle assumes the content of a dropped disk is not damaged or modified and preserves its membership in the diskgroup. Oracle introduces a new feature called ASM fast disk resync and does not automatically drop a disk from the diskgroup for a write failure.When a disk goes offline,Oracle now tracks all modified extents for a specified duration and keeps the disk membership in the diskgroup intact.
Once the disk is repaired or the temporary problem is resolved(that is, cable issue,controller, HBA, and so on),ASM can resynchronize the tracked extents that were modified during outage. In Oracle Database 11g, the time to recover from a perceived disk failure is directly relative to how many extents have changed during the outage.ASM can quickly resynchronize the changed extents on the failed disk with its surviving disks.
The potential increase in performance for the fast disk resync feature is proportional to the number of changed allocation units.Resync activities that take hours in Oracle Database 10g can perceivably be down to minutes in Oracle Database 11g.
Disk Repair Time
We’ll now talk about how the disk repair timer works.In Oracle Database 11g,the fast mirror resync feature is implemented using a grace period allotted to repair an offline disk. There is a new diskgroup attribute called disk_repair_time that allows you to specify the maximum amount of time before dropping a failed disk from the diskgroup.
The purpose of disk_repair_ time is to prevent dropping the disk from the diskgroup since a resynchronization operation is significantly less expensive than a rebalance operation.The default value of disk_repair_time is 3.6 hours,or 12,960 seconds.
The maximum allowable value for this attribute is 136 years.To take advantage of the fast disk resync feature,the compatibility attribute of the ASM diskgroup must be set to 188.8.131.52 or higher. The V$ASM_ATTRIBUTE view provides information about the compatibility level of the diskgroups:SQL> select name,compatibility
By default,the compatibility for diskgroup is set to 10.1. You need to advance this to 11.1 to take advantage of disk_repair_time. You cannot set the disk_repair_time attribute with the create diskgroup syntax.It can be set only with the alter diskgroup command.In this example, we’ll show how to advance the compatible.asm and compatible.rdbms attributes for the DATA diskgroup to 11.1 to take advantage of the repair timer:SQL> alter diskgroup data set attribute 'compatible.asm' = '11.1';
Diskgroup altered.SQL> alter diskgroup data set attribute 'compatible.rdbms'='11.1';
Now you can query the V$ASM_ATTRIBUTE view to confirm the settings:SQL> select name, value from v$asm_attribute; NAME VALUE
If you need to increase the duration of the disk_repair_time to five hours,you can change it with the following command:
You can specify this disk_repair_time unit in minutes (M or m) or hours(H or h).If not specified,the default is hours.As you can see here, you can also set the disk_repair_time at the minute granularity:
You can query the REPAIR_TIMER column to see the remaining time left in seconds before ASM drops an offline disk in the V$ASM_DISK or V$ASM_DISK_IOSTAT view.In addition, the disk resynchronization operation appears as a “SYNC” value in the OPERATION column of the V$ASM_OPERATION view.
Oracle’s background process,Diskgroup Monitor (GMON),wakes up every three minutes and checks all mounted diskgroups for offline disks.GMON will notify a slave process to increment their timer values(by three minutes)and initiate a dropfor the offline disks with the timer values exceeding their deadlines if it detects an offline disk.This is shown in the REPAIR_TIMER column of V$ASM_DISK.
Online and Offline Disks
Associated with this grace period attribute of the diskgroup,Oracle Database 11g ASM also provides the online option to the alter diskgroup disk command to initiate a disk resynchronization operation.This statement will copy all the extents that are marked as stale using the redundant copies:SQL> alter diskgroup disk data_0001 online;
Similar to onlining a disk,you can offline a disk with the alter disk group disk offline SQL command for preventive maintenance:SQL> alter diskgroup data offline disk data_0000 drop after 20 m;
You can accomplish this same onlining and offlining of diskgroups using the Enterprise Manager Database Console.From the Disk group main page for the specific diskgroup,you will notice the Online and Offline buttons, as displayed in Figure.
Figure ASM diskgroup Offline disk button
You can pick the disk in question and click the Offline button.This will direct you to the Disk Offline Confirmation page.You can choose the appropriate disk repair time and offline specified disk.Figure shows this example.
Figure ASM diskgroup offline disk
ASM will update the MOUNT_STATUS and MODE_STATUS columns as MISSING and OFFLINE states.Also, the REPAIR_TIMER column will start counting down to drop the disk from the diskgroup. The following example will query the V$ASM_DISK view to check the status of the disk and to look at the REPAIR_TIMER column to see the remaining time:select name, header_status,
Assuming the disk did not drop from the diskgroup,you can online the disk using alter disk group disk online once the maintenance is complete.
The following is an excerpt from the ASM alert.log file during the offline and online of the disk:SUCCESS: Advanced compatible.asm to 184.108.40.206.0 for grp 1
Once the disk issue is resolved,you can bring the disk back online using the online option to the alter diskgroup command:
SQL> alter diskgroup data online disk data_0000;
You should see now see that REPAIR_TIMER is set back to 0 and all the status of the disks are back to ONLINE.You can query the V$ASM_DISK view to confirm that the disk is ONLINE,as displayed here:select name, header_status, mount_status,
SAN administrators and SAs may receive alerts from the SAN monitoring tools that a particular disk may go bad soon. For such situations, disks will have to be replaced and measures will need to take place so that it does not become an outage.
Oracle 11g Related Interview Questions
|Oracle 10g Interview Questions||Oracle 9i Interview Questions|
|Oracle 8i Interview Questions||Informatica Interview Questions|
|PL/SQL Interview Questions||Oracle 11g Interview Questions|
|SQL Interview Questions||Oracle apps Interview Questions|
|Sybase Interview Questions||Oracle Apps ERP Interview Questions|
|Oracle 7.3 Interview Questions||Oracle Access Manager Interview Questions|
|Oracle Application Framework Interview Questions||Oracle Apps DBA Interview Questions|
Oracle 11g Related Practice Tests
|Oracle 10g Practice Tests||Oracle 9i Practice Tests|
|Oracle 8i Practice Tests||Informatica Practice Tests|
|PL/SQL Practice Tests||Oracle 11g Practice Tests|
|SQL Practice Tests||Oracle apps Practice Tests|
|Sybase Practice Tests||Oracle Apps ERP Practice Tests|
|Oracle 7.3 Practice Tests|
Oracle 11g Tutorial
Installing, Upgrading, And Managing Change
Database Diagnosability And Failure Repair
Backup And Recovery
All rights reserved © 2018 Wisdom IT Services India Pvt. Ltd
Wisdomjobs.com is one of the best job search sites in India.