Oracle VM Manager (OVMM): How To Check/Verify If There Is Corruption In OVM Manager Database (DB) From OVM CLI?

Below are steps to check/verify of there is corruption in OVM Manager Database (DB) from OVM CLI

1. Login to Host running Oracle VM Manager as "root" user.

2. Login to OVM shell prompt by running below command.       

/usr/bin/ovm_shell.sh -u admin -p <password>

     
In above command replace <[password> with the OVMM "admin" user password.  After successful login you will be logged into ">>>" OVMM shell prompt.
   
3. Run below commands in same order in OVMM shell prompt.

om = OvmClient.getOvmManager()

f = om.getFoundryContext()

f.fixupScan()

    
If there is corruption in OVM Manager DB, you will see outputs like ONF, inconsistencies etc. once you run "f.fixupScan()". If there is no corruption the output will be empty.

Products to which Article Applies

Oracle VM Manager (Versions 3.3.X & Above)

Read more "Oracle VM Manager (OVMM): How To Check/Verify If There Is Corruption In OVM Manager Database (DB) From OVM CLI?"

Linker Article: Oracle VM Manager (OVMM): OVM Database Related Articles (DB Corruption, Table/Queue Size, How To Docs etc.)

• Oracle VM Manager (OVMM): How To Check If there is Database Corruption in OVM Manager?
• Oracle VM Manager: How To Check OVM_STATISTIC Table File Size and Records For Troubleshooting OVMM Slowness Issue?

 

Read more "Linker Article: Oracle VM Manager (OVMM): OVM Database Related Articles (DB Corruption, Table/Queue Size, How To Docs etc.)"

Oracle VM Manager (OVMM): How To Check OVM_STATISTIC Table File Size and Records For Troubleshooting OVMM Slowness Issue?

In some cases we will see slowness from OVM Manager where jobs will take forever to complete or will hang. These kind of issues can happen if Oracle VM Manager statistics table file size grows huge in size or if there are millions of records in Oracle VM Manager statistics table.

Steps in this article will provide steps on how to validate OVM statistics table and number of records in OVM statistics table.

Verify OVM statistics table file size

Below command can be executed to check the file size of OVM statistics table file size.

ls -hlrt /u01/app/oracle/mysql/data/ovs/OVM_STATISTIC.ibd

Usually above OVM_STATISTIC.ibd file size should be few GB in size (less than 5 GB) if OVM Manager is healthy.

Verify the number of records in OVM statistics table file

Follow below steps to check the records in OVM_STATISTIC statistics table

a. Run below command. When prompted for password enter the password, default is welcome1.

mysql ovs -u ovs -p -S /u01/app/oracle/mysql/data/mysqld.sock

You will be logged into SQL prompt.

b. Run below command in SQL prompt.

select count(*) from OVM_STATISTIC;

Usually records in OVM_STATISTIC.ibd table should be less than million if OVM Manager is healthy.

Products to which Article Applies

Oracle VM Manager - All versions

Additional References

https://docs.oracle.com/cd/E50245_01/

Read more "Oracle VM Manager (OVMM): How To Check OVM_STATISTIC Table File Size and Records For Troubleshooting OVMM Slowness Issue?"

Oracle VM Manager (OVMM): How To Check If there is Database Corruption in OVM Manager?

We can check following to see if there is corruption in Oracle VM Manager Database.

1. Look for ONF (Object Not Found) errors in OVM DB
2. Validate if Daily AutoFullBackup DB backups stopped working.
3. Look for Object Not Found (ONF) and Cluster is null errors in OVM Manager Admin Server logs.

Below are detailed steps for validating above points to determine if there is OVM DB corruption.

1) Look for ONF (Object Not Found) errors in OVM DB

For this login to the Host where OVM Manager is running and follow below steps to check if there are ONF errors in OVM DB.

a. Login to EMOC Control vserver which runs OVMM as root user.

b. Login to OVM shell prompt by running below command.

/usr/bin/ovm_shell.sh -u admin -p <password>

In above command replace <password> with the OVMM admin user password. You will be logged into ">>>" OVMM shell prompt.

c. Run below commands in same order in OVMM shell prompt.

om = OvmClient.getOvmManager()

f = om.getFoundryContext()

f.fixupScan()

If there is no ONF errors we should see blank output []. If we see any output when running above steps, it means that there is corruption and inconsistencies in OVM DB.

2) Validate if Daily AutoFullBackup DB backups stopped working.
Run below command to validate if there are AutoFullBackup DB backups are happening regularly.

ls -ldt /u01/app/oracle/mysql/dbbackup/Auto*

Please note above is the default OVM DB autobackups directory. If you have changed the default directory make changes to above command as needed.

3) Check if there are any "cluster is null" or "ObjectNotFound" errors in OVM Manager Admin Server logs

For this login to the Host where you have OVM Manager running and run below command.

egrep -iR "cluster is null|ObjectNotFound" /u01/app/oracle/ovm-manager-3/domains/ovm_domain/servers/AdminServer/logs/AdminServer*

If you see ObjectNotFound or "cluster is null" errors as follows, that means that there is corruption in OVM Manager DB.

<Exception in archiver moving jobs to jpa store: No such object (level 1), cluster is null: <XXXX>

<Day> <Mon DD Time> com.oracle.odof.exception.ObjectNotFoundException: No such object (level 1), cluster is null: <XXXX>

Above is the default OVM Manager Admin Server log directory in OVM 3.3 & higher versions. In some versions of OVM Manager (old versions) the Admin Server log directory may be different.

Products to which Article Applies

Oracle VM Manager versions 3.3 & Above

Additional References

https://docs.oracle.com/cd/E50245_01/

Read more "Oracle VM Manager (OVMM): How To Check If there is Database Corruption in OVM Manager? "