Primary key creation using dbms_metadata ddl and after impdp

>> Wednesday, 5 June 2013

I had a problem where code stopped working the same way after I had recreated a primary key for the developers.  Initially they asked me to recreate an index, which I did. Then they asked me for a primary key so I created it using the index.  Their code started to error where it had not done before..

What they should have asked for was a primary key not a primary key using an existing index. There is a big difference in the way they behave when you delete the key. 

With a primary key that created it's own index, if you drop the key, the index goes with it.

If you create a primary key on an existing index, and then drop the key, the index stays in place, just the constraint gets dropped.

I'll show you how it works, but for a really great explanation, I also recommend you go have a look at http://viralpatel.net/blogs/understanding-primary-keypk-constraint-in-oracle/ as it explains it very well.

Here we go:

Create primary key allowing Oracle to create index

drop table tbl_test;
CREATE TABLE tbl_test ( col_1 NUMBER,col_2 NUMBER);
ALTER TABLE tbl_test ADD CONSTRAINT tbl_test_pk PRIMARY KEY(col_1);

select index_name, table_name from dba_indexes where index_name='TBL_TEST_PK';
INDEX_NAME                     TABLE_NAME
------------------------------ ------------------------------
TBL_TEST_PK                    TBL_TEST

SELECT constraint_name, constraint_type, index_name   FROM user_constraints
  WHERE table_name = 'TBL_TEST';
CONSTRAINT_NAME                C INDEX_NAME
------------------------------ - ------------------------------
TBL_TEST_PK                    P TBL_TEST_PK


This statement is the key bit of information about whether it will delete the index with the key drop or not, it;s explanation is at Jonathon Lewis' Scratchpad http://jonathanlewis.wordpress.com/2012/04/19/drop-constraint/

select do.object_name, do.object_id, bitand(ind$.property,4096), ind$.property
from   sys.ind$    ind$, dba_objects do 
where  do.owner='SYSTEM' 
and do.object_name = 'TBL_TEST_PK'
and do.object_id=ind$.obj#;
OBJECT_NAME
--------------------------------------------------------------------------------
 OBJECT_ID BITAND(IND$.PROPERTY,4096)   PROPERTY
---------- -------------------------- ----------
TBL_TEST_PK
     50284                       4096       4097

Dropping the primary key means the automatically created index also gets dropped.

ALTER TABLE tbl_test DROP PRIMARY KEY;
Table altered.
select index_name, table_name from dba_indexes where index_name='TBL_TEST_PK';
no rows selected

SELECT constraint_name, constraint_type, index_name   FROM user_constraints
  WHERE table_name = 'TBL_TEST';
no rows selected

select do.object_name, do.object_id, bitand(ind$.property,4096), ind$.property
from   sys.ind$    ind$, dba_objects do 
where  do.owner='SYSTEM' 
and do.object_name = 'TBL_TEST_PK'
and do.object_id=ind$.obj#;
no rows selected



Create a primary key using an existing index

drop table tbl_test2;
CREATE TABLE tbl_test2 ( col_1 NUMBER, col_2 NUMBER);
CREATE UNIQUE INDEX idx_col_1 ON tbl_test2(col_1);
ALTER TABLE tbl_test2 ADD CONSTRAINT tbl_test2_pk PRIMARY KEY(col_1) USING
INDEX idx_col_1;

select index_name, table_name from dba_indexes where index_name='IDX_COL_1';
INDEX_NAME                     TABLE_NAME
------------------------------ ------------------------------
IDX_COL_1                      TBL_TEST2

SELECT constraint_name, constraint_type, index_name   FROM user_constraints
  WHERE table_name = 'TBL_TEST2';
CONSTRAINT_NAME                C INDEX_NAME
------------------------------ - ----------------
TBL_TEST2_PK                   P IDX_COL_1

select do.object_name, do.object_id, bitand(ind$.property,4096), ind$.property
from   sys.ind$    ind$, dba_objects do 
where  do.owner='SYSTEM' 
and do.object_name='IDX_COL_1'
and do.object_id=ind$.obj#;
OBJECT_NAME
--------------------------------------------------------------------------------
 OBJECT_ID BITAND(IND$.PROPERTY,4096)   PROPERTY
---------- -------------------------- ----------
IDX_COL_1
     50286                          0          1

dropping the primary key will not delete the index

ALTER TABLE tbl_test2 DROP PRIMARY KEY;

select index_name, table_name from dba_indexes where index_name='IDX_COL_1';
INDEX_NAME                     TABLE_NAME
------------------------------ ------------------------------
IDX_COL_1                      TBL_TEST2

SELECT constraint_name, constraint_type, index_name   FROM user_constraints
  WHERE table_name = 'TBL_TEST2';
no rows selected


select do.object_name, do.object_id, bitand(ind$.property,4096), ind$.property
from   sys.ind$    ind$, dba_objects do 
where  do.owner='SYSTEM' 
and do.object_name='IDX_COL_1'
and do.object_id=ind$.obj#;
OBJECT_NAME
--------------------------------------------------------------------------------
 OBJECT_ID BITAND(IND$.PROPERTY,4096)   PROPERTY
---------- -------------------------- ----------
IDX_COL_1
     50286                          0          1



So now we understand how dropping the primary key affects the index it is based on, we know it makes a big difference how it is actually created.

Now look at what happens when the constraints are imported:

drop table tbl_test;
CREATE TABLE tbl_test ( col_1 NUMBER,
 col_2 NUMBER);
ALTER TABLE tbl_test ADD CONSTRAINT tbl_test_pk PRIMARY KEY(col_1);

drop table tbl_test2;
CREATE TABLE tbl_test2 ( col_1 NUMBER, col_2 NUMBER);
CREATE UNIQUE INDEX idx_col_1 ON tbl_test2(col_1); 
ALTER TABLE tbl_test2 ADD CONSTRAINT tbl_test2_pk PRIMARY KEY(col_1) USING INDEX idx_col_1;


select do.object_name, do.object_id, bitand(ind$.property,4096), ind$.property
from   sys.ind$    ind$, dba_objects do 
where  do.owner='SYSTEM' 
and do.object_name in ('TBL_TEST_PK','IDX_COL_1')
and do.object_id=ind$.obj#;
OBJECT_NAME
--------------------------------------------------------------
 OBJECT_ID BITAND(IND$.PROPERTY,4096)   PROPERTY
---------- -------------------------- ----------
IDX_COL_1
     50290                          0          1

TBL_TEST_PK
     50288                       4096       4097


CREATE DIRECTORY dpump_dir2 AS '/home/oracle';

vi myparfile

CONTENT=ALL
DIRECTORY=DPUMP_DIR2
REUSE_DUMPFILES=Y
TABLES=
system.tbl_test
system.tbl_test2

expdp system dumpfile=mytest.dmp logfile=mytest.log parfile=myparfile

drop table tbl_test;
drop table tbl_test2;

impdp system full=Y content=ALL directory=DPUMP_DIR2 dumpfile=mytest.dmp logfile=mytestimp.log

The tables, indexes and constraints are back:

select index_name, table_name from dba_indexes where index_name='TBL_TEST_PK';
INDEX_NAME                     TABLE_NAME
------------------------------ ------------------------------
TBL_TEST_PK                    TBL_TEST

SELECT constraint_name, constraint_type, index_name   FROM user_constraints  WHERE table_name = 'TBL_TEST';
CONSTRAINT_NAME                C INDEX_NAME
------------------------------ - ------------------------------
TBL_TEST_PK                    P TBL_TEST_PK

select index_name, table_name from dba_indexes where index_name='IDX_COL_1';
INDEX_NAME                     TABLE_NAME
------------------------------ ------------------------------
IDX_COL_1                      TBL_TEST2

SELECT constraint_name, constraint_type, index_name   FROM user_constraints  WHERE table_name = 'TBL_TEST2';
CONSTRAINT_NAME                C INDEX_NAME
------------------------------ - ------------------------------
TBL_TEST2_PK                   P IDX_COL_1


So far so good, bit now look:


select do.object_name, do.object_id, bitand(ind$.property,4096), ind$.property
from   sys.ind$    ind$, dba_objects do 
where  do.owner='SYSTEM' 
and do.object_name in ('TBL_TEST_PK','IDX_COL_1')
and do.object_id=ind$.obj#;

OBJECT_NAME
-------------------------------------------------------------------------------
 OBJECT_ID BITAND(IND$.PROPERTY,4096)   PROPERTY
---------- -------------------------- ----------
IDX_COL_1
     50335                          0          1

TBL_TEST_PK
     50334                          0          1



So this time the index is NOT dropped in either type of primary key when the primary key is dropped:

ALTER TABLE tbl_test DROP PRIMARY KEY;

select index_name, table_name from dba_indexes where index_name='TBL_TEST_PK';
INDEX_NAME                     TABLE_NAME
------------------------------ ------------------------------
TBL_TEST_PK                    TBL_TEST

SELECT constraint_name, constraint_type, index_name   FROM user_constraints  WHERE table_name = 'TBL_TEST';
no rows selected

ALTER TABLE tbl_test2 DROP PRIMARY KEY;
Table altered.

select index_name, table_name from dba_indexes where index_name='IDX_COL_1';

INDEX_NAME                     TABLE_NAME
------------------------------ ------------------------------
IDX_COL_1                      TBL_TEST2

SELECT constraint_name, constraint_type, index_name   FROM user_constraints  WHERE table_name = 'TBL_TEST2';
no rows selected


Impdp simply creates the keys as if they were created on existing indexes, it doesn't recreate them as they were built originlally.  I think this sucks.

dbms_metadata doesn't get it right either:

dbms_metadata also doesn't understand that primary keys have been created differently, but it treats them the opposite way to datapump:
                                                      
        
Create an inline primary key
                                                                      
 create table test (
 snorm   number,
 snurm   varchar2(40),
snerm   date,
 snirm   varchar2(4),
  constraint test_pk primary key(snorm));


select dbms_metadata.get_ddl ('TABLE','TEST','SYSTEM') from dual;
                                                                              
  CREATE TABLE "SYSTEM"."TEST"                                               
   ( "SNORM" NUMBER,                                                           
 "SNURM" VARCHAR2(40),                                                         
 "SNERM" DATE,                                                                 
 "SNIRM" VARCHAR2(4),                                                          
  CONSTRAINT "YINK_PK" PRIMARY KEY ("SNORM")                                   
  USING INDEX PCTFREE 10 INITRANS 2 MAXTRANS 255                               
  TABLESPACE "USERS"  ENABLE                                                   
   ) SEGMENT CREATION DEFERRED                                                 
  PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255                                
 NOCOMPRESS LOGGING                                                            
  TABLESPACE "USERS"                                                           
                                                                               
 select dbms_metadata.get_dependent_ddl('INDEX','TEST','SYSTEM') from dual;
                                                                              
  CREATE UNIQUE INDEX "SYSTEM"."TEST_PK" ON "SYSTEM"."TEST" ("SNORM")      
  PCTFREE 10 INITRANS 2 MAXTRANS 255                                           
  TABLESPACE "USERS"                                                           
                                                                               

Create a primary key on an existing index:

create table test2(
   snorm   number,
   snurm   varchar2(40),
   snerm   date,
   snirm   varchar2(4));

create index test2_idx on test2(snorm);

alter table test2 add constraint test2_pk primary key(snorm) using index test2_idx;

 select dbms_metadata.get_ddl('TABLE','TEST2','SYSTEM') from
                                                                              
  CREATE TABLE "SYSTEM"."TEST2"                                               
   ( "SNORM" NUMBER,                                                           
 "SNURM" VARCHAR2(40),                                                         
 "SNERM" DATE,                                                                 
 "SNIRM" VARCHAR2(4),                                                          
  CONSTRAINT "TEST2_PK" PRIMARY KEY ("SNORM")                                   
  USING INDEX PCTFREE 10 INITRANS 2 MAXTRANS 255 COMPUTE STATISTICS            
  TABLESPACE "USERS"  ENABLE                                                   
   ) SEGMENT CREATION DEFERRED                                                 
  PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255                                
 NOCOMPRESS LOGGING                                                            
  TABLESPACE "USERS"                                                           
                                                                               
SQL> select dbms_metadata.get_dependent_ddl('INDEX','TEST2','SYSTEM') from dual;
                                                                              
  CREATE INDEX "SYSTEM"."TEST2_IDX" ON "SYSTEM"."TEST2" ("SNORM")            
  PCTFREE 10 INITRANS 2 MAXTRANS 255 COMPUTE STATISTICS                        
  TABLESPACE "USERS"                                                           
                                                                               


dbms_metadata produces the same ddl for both despite them being created very differently.

So using dbms_metadata to produce create database scripts could cause issues.  And incase you are wondering Toad does the same thing.

So if developers code relies on the index being retained or not when they drop constraints (as they often do before large data loads) then recreating uat databases using these methods could store up a heap of trouble later.


*Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit


Further reading:  https://nzdba.wordpress.com/tag/ind/

Read more...

11.2.0.3 upgrade ORA-27248: sys.dra_reevaluate_open_failures is running

>> Tuesday, 29 January 2013

I was doing a simple upgrade of Oracle RDBMS EE 11.2.0.2 to 11.2.0.3 using dbua.  It should have been a no brainer.  Except part way through I got the error ORA-27248: sys.dra_reevaluate_open_failures* is running.

I had to abort the upgrade which then meant I had to do a full rman recovery and start again.

The problem:  neither the pre-upgrade notes (or dbua) cover a check to ensure there are no scheduler jobs running.  Although even if you did check manually prior to hitting the 'yup upgrade' button, you could still be unlucky and the maintenance window start and the oracle internal jobs get going. 

Open migrate doesn't care, it allows these jobs to carry on running and the dbua doesn't check for them.

Why?  This was Oracle's answer:

If the database is running in restricted mode then no jobs will run (unless
you are using 11g and use the ALLOW_RUNS_IN_RESTRICTED_MODE attribute).
To check this use
SQL> select logins from v$instance ;
If logins is restricted you can disable the restricted mode using
SQL> ALTER SYSTEM DISABLE RESTRICTED SESSION;
ref: http://docs.oracle.com/cd/E11882_01/appdev.112/e16760/d_sched.htm
I'm not sure why they think this would even begin to resolve this issue, infact if the upgrade was done in restructed mode then surely I wouldn't have had the issue.

The job sys.DRA_REEVALUATE_OPEN_FAILURES is created by default so that allow_runs_in_restricted_mode is false, the default is false.  It shouldn't run if the database is opened restricted mode.

So I must conclude that STARTUP UPGRADE is not putting the database into restricted mode and scheduled jobs will continue to run.

The pre-upgrade check notes make no mention of needing to do anything with scheduled jobs, but I have added a check into my process:

Select * from v_$scheduler_running_jobs;
Select * from dba_scheduler_running_jobs;

If there are jobs running then chances are dbua is going to abort *shrug*  probably *shrug*

After pressing Oracle for a while longer, they suggested this:

MY CAVEAT: DO NOT RUN SQL BELOW BEFORE READING AHEAD, DO NOT RUN THIS IF YOU HAVE THIS PROBLEM.
startup in restricted mode
startup restrict;
2. disable all windows
select 'exec DBMS_SCHEDULER.DISABLE('''||WINDOW_NAME||''');' from ALL_SCHEDULER_WINDOWS where ENABLED='TRUE';
3. close all windows
select 'exec DBMS_SCHEDULER.CLOSE_WINDOW('''||WINDOW_NAME||''');' from ALL_SCHEDULER_WINDOWS where ACTIVE='TRUE';
4. stop running & disable jobs
CASE WHEN state !='RUNNING' THEN 'begin' || CHR(10) || 'DBMS_SCHEDULER.STOP_JOB ('''||OWNER||'.'||JOB_NAME||''', TRUE); ' || CHR(10) || 'EXCEPTION WHEN OTHERS THEN NULL;' || CHR(10) || 'END; ' || CHR(10) || '/' || CHR(10) || 'exec DBMS_SCHEDULER.DISABLE ('''||OWNER||'.'||JOB_NAME||''', TRUE );'
ELSE
'exec DBMS_SCHEDULER.DISABLE ('''||OWNER||'.'||JOB_NAME||''', true);'
end

I'm  not convinced this is a good approach.  If you disable jobs you need to reenable them later. No?  So which jobs did you just disable?  No, I've no idea either?!  Which Windows did you just disable? Again, I've no idea! Which of your windows were already disabled?

Teflon Oracle Support.

I pressed Oracle again and they agreed that there needs to be a preupgrade process to check for running jobs before upgrading.

Until they update their documentation, I recommend you run :

Select * from v_$scheduler_running_jobs;
Select * from dba_scheduler_running_jobs;

And don't run your upgrade if you have running jobs.  If you do have running jobs and want to get rid of them then yes, I would do:

shutdown,
startup restrict,
select WINDOW_NAME, ENABLED  from ALL_SCHEDULER_WINDOWS where ENABLED='TRUE';

Make a note of those windows and disable them.

exec DBMS_SCHEDULER.DISABLE('WINDOW_NAME'); DBMS_SCHEDULER.DISABLE('WINDOW_NAME');
              at the end of the upgrade you need to remember to re-enable them.
              exec DBMS_SCHEDULER.ENABLE('WINDOW_NAME');


 If there are any Active windows you need to close them:

select WINDOW_NAME, ACTIVE from ALL_SCHEDULER_WINDOWS where ACTIVE='TRUE';

Make a note of those windows and disable them.

exec DBMS_SCHEDULER.CLOSE_WINDOW('WINDOW_NAME');WINDOW_NAME');
                     I expect these would become active again when get to their next active time and the window is
                     enabled.

check whether any jobs are running:

Select * from v_$scheduler_running_jobs;
Select * from dba_scheduler_running_jobs;

You could at this point stop the jobs but I would expect a restart to stop the jobs and because the window is disbled they ought to not restart. (I need to find time to test this.)

Shutdown immediate;
startup normal;

Select * from v_$scheduler_running_jobs;
Select * from dba_scheduler_running_jobs;

this must return no rows

and now you can follow the upgrade process.

I'm totally bemused, confused why no one else appears to have run into this problem or if they have, why have they not put something 'out there' in discussion.  I am still wondering whether I'm barking up the wrong tree SO IF YOU DO HIT THIS ISSUE PLEASE RAISE YOUR OWN SR AND GET SUPPORT FROM ORACLE but you may want to use these notes as a useful pointer.


*If you care the description for dra_reevaluate_open_failures is 're-evaluates open failures for the Database Recovery Advisor (used for assistance in case of loss or corruption of datafiles, redo log files or controlfiles).  Executes dbms_ir.reevaluateopenfailures.'  This isn't really relevant to this post as it could potentially occur with any job.

Read more...

UKOUG 2012 Birmingham

>> Saturday, 8 December 2012

I had the fantastic opportunity to go to the UKOUG 2012 for a day and I can't thank the wonderful Oracle developer who arranged it for me enough.

What an incredible opportunity.  If you didn't read the reasons I gave for being a member of the  UK Oracle User Group then you should.

I had a day of intense learning and also discover a number of experts and Ace's completely new to me, but with so much to share and so many new websites and blogs to read.   As a full member you can spent 3 days here in solid learning.  But there is so much more.

There is an incredible feeling of motivation, everyone here is interested, wants to learn, wants to become the best they can be at their job.  You can stand and have a lengthy conversation with a stranger about your SGA issue, or whether huge pages is the way to go and not stand out as odd.  There were probably a 1000 of more people there but with so many auditoriums, stands and activities, there were no bun fights for seats, there was plenty of space, plenty of food and plenty of options.

In fact I recommend you plan a little bit of downtime into your day because you could fill every second for 8am to 11pm and not stop.

I wish I had taken more photos but I was just too busy learning, writing, and socialising.  I met a lot of people I talk to on line face to face.  It really helps to bond the self help community to actually meet people and have a moment to discuss Oracle face to face.  It is quite a strange feeling to recognise someone by their avatar as they walk past you and tweeting between presentations I discovered how many of the people I see scrolling past my eyes daily were there also.

Yet again I came back to work totally motivated, wanting to push a little bit harder, self-learn a little bit more.

 Although seeing Tom
 and playing scalextric

may have had a little to do with it.
 I have to say that discovering  Kerry Osborne via Controlling Execution Plans (without touching the code) and the Cache 'n' Query presentation by Jaromir Nemic actually stand out as high points of my day.

If you are a member and don't use your access to the main conference you should, and for non-members, well it's time to join.




Read more...

Oracle UKOUG Why It's worth joining.

>> Wednesday, 10 October 2012

Today I was lucky enough to be able to attend a UK Oracle User Group Database Server SIG meeting.  It's been a few years since I went to one of these.

My company doesn't subscribe to any user groups any more, it's their policy.  I suspect because the person holding the budget has no idea at all what happens at one of these and has a false impression of a time wasting out of office jolly and god forbid a good time could be had by any dungeon living IT technician.

The value of these SIG days should not be underplayed.

Today I found out how to use snoop a step on from strace/truss for suspect network issues,  I discovered the ins and outs of working from home in the UK whilst outsourcing for a Canadian company, how to compare AWR reports, looking into high buffer reads and about IBMs Netezza. All this was over coffee breaks and lunch.  Networking, sharing experiences and meeting like minded people is a great part of the day. 

But let's put the fun aside:

The Support update was a solid reminder of the needs for patching, what sorts of issues are out there etc.  But all this is available in metalink, I hear you cry.  Yep, Metalink is sat there whirring away in the background but whilst I'm in the office up to my neck in issues and blaming the infrastructure, it is good for me to be mindful of the types of major faults Oracle are still mopping up and these presentations do that for me.

Before stepping into the building I knew nothing at all about Exadata.  I'd read a little, I've stood on the sidelines of social media conversation but I didn't really know any useful basics at all.  45 minutes of e-DBA's Jason Arneil (@jarneil) and I have a grounding of the basics, enough to allow me to read technical documents without just hearing white noise.    I know I am unlikely to ever get formal training on this even if we were to implement them.  As far as I am concerned that session justified the cost of the day.

But wait there's more, Pete Finnigan did a session on Secure Coding PL/SQL.  I've seen him present on profiles, roles, inherited rights  invoker rights before and if you haven't, you should.  But this was a step on into how to start to protect you sql procedures from sql injection attacks, how to look for the holes in your pl/sql and how to start closing them.

There were sessions on  Edition Based Redefinition which is something I was totally unaware of and GoldenGate something I had heard of but had no understanding of.  I think these 2 things are well worth me being fully aware of especially knowing that GoldenGate can be short term licensed.  I now have another option to propose when I need to upgrade a critical 24/7.

But the grand finale was Jonathon Lewis presenting on Exadata compression, storage indexes and Exadata's use of indexes.  None of the presentation was directly relevant to anything I'm doing right now and by the time I'm fortunate enough to come into contact with Exadata I'm likely to have forgotten the detail, that isn't why it matters.  This is the reason why:

                                 MOTIVATION

Being a DBA can sometimes seem like a firefighting daily grind, a battle with developers, a battle with infrastructure design, a non IT driven management that don't understand the technical detail or really care.  In short it can be a drag and I start to wonder why I do it at all.  Jonathon's presentation got back to the core of what I really love, why I ended up here in the first place.  Talking about compression headers is interesting, working out how many rows you can squeeze into 32k is fun and I'm still wondering why Oracle picked 32,759 rows for a compression unit.  The process behind a query dances on the powerpoint and Jonathon at points appeared to be dancing in front of it! Why should the predicates matter...look at that hint... I didn't know that hint.. wow look at THAT hint. He over ran and as so many of us wanted to see the end, he happily stayed behind and finished the presentation for us.  He clearly enjoys the dancing data too.

Companies that don't pay for this membership are clearly very short sighted.  You need real technicians to be enthused about what they do.  If they are motivated, they will do it well, they'll even do it in their own time, they'll want to do it.  The cost of membership to a company to buy that type of staff motivation is as cheap as chips.

I am now seriously considering a personal membership for next year not so chip cheap but worth it.  I know my company's policy is not for turning but I need this sort of regular infusion of update, refresh and reinvigoration to make me the best DBA I can be.

I was rather pleased at the end of the day to have another delegate come up to me and say "What company do you work for? They must be good to be employing such intelligent people that can ask such good questions"   Bless him,  I'm not sure he understood I ask so many questions because I don't have the answers not because I was trying to bring up good points!

I'm sure if you've come here via a search on UKOUG membership you will know that it includes many privileges beyond SIG attendance including discount on Oracle University training. Go and have a look at their website for more details.

Read more...

v$object_usage invisible index used

>> Monday, 20 August 2012

I had a number of indexes on a table I made invisible, they were large indexes on a table that seemed totally useless, many were duplicates and as they accounted a large amount of the disk I was keen to see them gone and get some space back.  The application developers were happy for me to give it a try when I explained how making an index invisible in Oracle 11.2 works.

Invisible is not the same as unusable.  An invisible index is totally maintained, you are simply telling the CBO not to use it, and if you find you hit a performance problem you can quickly make it visible again without having to rebuild it.

alter index schema_owner.my_index_name invisible;
alter index schema_owner.my_index_name visible;

Dropping the indexes of this size though would be a huge issue for me to recreate so I wanted to be absolutely sure they weren't being used before I did that, so I decided to monitor them for a while.  This is easy too:

alter index schema_owner.my_index_name monitoring usage;

Now have a look at V$object_usage (as the object owner - if you do it as any other user you won't get any data returned)

select * from v$object_usage where index_name='MY_INDEX_NAME';
MY_INDEX_NAME MY_TABLE_NAME  YES NO date_and_time_monitoring_started

The USED=NO is good, this means Oracle hasn't used my index. But every morning when I got in the used had become 'YES'.

No matter how hard I tried I could not catch the sql that was causing the field to be changed

The sql pool was showing that DBMS_STATS was using a use_invisible_indexes hint but surely Oracle would have factored that in. This was the proof they had:

Alter system flush shared pool;

select count(*)
from v$sqlarea a, v$sql_plan p
where p.hash_value=a.hash_value
and p.address=a.address
and p.object_name='MY_INDEX_NAME';
zero rows returned

select index_name, index_type, visibility  from dba_indexes where table_name='MY_TABLE_NAME';
MY_INDEX_NAME NORMAL VISIBLE
alter index schema_owner.my_index_name invisible;

select index_name, index_type, visibility  from dba_indexes
 where table_name='MY_TABLE_NAME';
MY_INDEX_NAME NORMAL INVISIBLE
alter index schema_owner.my_index_name nomonitoring usage;
alter index schema_owner.my_index_name monitoring usage;

(as schema owner)
select USED from v$object_usage where index_name='MY_INDEX_NAME';
NO

select index_name, last_analyzed  from dba_indexes where table_name='MY_TABLE_NAME';
MY_INDEX_NAME 29/07/2012 15:46:08
exec dbms_stats.gather_table_stats('SCHEMA_OWNER','MY_TABLE_NAME',cascade => TRUE);
PL/SQL procedure successfully completed.

select index_name, last_analyzed from dba_indexes where table_name='MY_TABLE_NAME';
MY_INDEX_NAME 09/08/2012 12:44:11
(as schema owner)
select USED from v$object_usage where index_name='MY_INDEX_NAME';
NO

select count(*)
from v$sqlarea a, v$sql_plan p
where p.hash_value=a.hash_value
and p.address=a.address
and p.object_name='MY_INDEX_NAME';
1 row returned

select a.sql_text
from v$sqlarea a, v$sql_plan p
where p.hash_value=a.hash_value
and p.address=a.address
and p.object_name='MY_INDEX_NAME';
select /*+  no_parallel_index(t, "MY_INDEX_NAME")  dbms_stats cursor_sharing_exact use_weak_name_resl dynamic_sampling(0) no_monitoring no_substrb_pad  no_expand use_invisible_indexes index(t,"MY_INDEX_NAME") */ count(*) as nrw,count(distinct sys_op_lbid(36991,'L',t.rowid)) as nlb,null as ndk,sys_op_countchg(substrb(t.rowid,1,15),1) as clf from "SCHEMA_OWNER"."MY_TABLE_NAME" t where ("OSUSER" is not null) and (TBL$OR$IDX$PART$NUM........etc


So Oracle is using the index when dbms_stats is run but v$object_usage is not updated.

I was stuck here for a long time, every day I reset monitoring, every night it was updated but there was nothing in the sql pool to give me an idea as to why.  In the hope that it was being pushed out of the sql area I set a cron job to run every 5 minutes over night that ran the select sql_text query if USED in v$object_usage had changed to yes. I could see the point it changed but still I was getting no sql other than that from dbms_stats.

The application developers were adamant they did not use hints in their code anywhere. 

So what on earth was using it?  Then a number of weeks after I had been pulling my hair out I got cc'd on an email from the schedulers to the developers about a job that was over-running and the sql output they attached from the job named my index.  In fact being aware that I was 'working' on the index I was blamed for making the job over run.

It turns out that every night as a part of the batch the application rebuilt every partition of every index belonging to this table.  They weren't checking whether the index was visible or not.  I wouldn't expect them to check for that to be fair, what I still cannot fathom though is why they were rebuilding every subpartition of a local index on a partitioned table every night.  Local indexes manage themselves really.  Drop the table partition, the index partition goes too.  I can only think it's a code hangover from pre local indexes, unless you want to tell me different.

So here's the proof it was an index rebuild causing my headache:

alter system flush shared pool;

select count(*)
from v$sqlarea a, v$sql_plan p
where p.hash_value=a.hash_value
and p.address=a.address
and p.object_name='MY_INDEX_NAME';
zero rows returned


select index_name, index_type, visibility from dba_indexes where table_name='MY_TABLE_NAME';
MY_INDEX_NAME NORMAL VISIBLE
alter index schema_owner.my_index_name invisible;

select index_name, index_type, visibility from dba_indexes
where table_name='MY_TABLE_NAME';
MY_INDEX_NAME NORMAL INVISIBLE
alter index schema_owner.my_index_name nomonitoring usage;
alter index schema_owner.my_index_name monitoring usage;

(as schema owner)
select USED from v$object_usage where index_name='MY_INDEX_NAME';
NO

alter index schema_owner.my_index_name REBUILD SUBPARTITION my_subpartition_name;

(as schema owner)
select USED from v$object_usage where index_name='MY_INDEX_NAME';
YES

select count(*)
from v$sqlarea a, v$sql_plan p
where p.hash_value=a.hash_value
and p.address=a.address
and p.object_name='MY_INDEX_NAME';
zero rows returned



Oracle have logged an enhancement request for v$object_usage to include a column that reports the action that causes USED to be altered, this would have saved me a whole lot of head scratching.

Read more...

Oracle Forum Hostility

>> Saturday, 18 August 2012

Thomas Kyte has written a very good post on The Keys to Oracle, the best starting points for a journey to a decent working knowledge of Oracle.  But he also recommends a great way to learn is to start asking questions on http://otn.oracle.com/forums. On this point I have to disagree with him.

I only use Oracle forums when a search engine leads me there.  They can be good as catalysts for me to get to the root of my problem but all too often I am appalled by responses I see to questions asked.

Now I know I have to be fair here, there are some very helpful and dedicated people that work hard to support and help people in the forums.  I regularly see the same names giving some great support and answers. If it weren't for them the forums wouldn't hold so much of the great information they do.

But even with my many years of Oracle experience, I am too scared to ask a question in the forums.  There are just too many sarcastic, unhelpful responses given that I know I would find very upsetting if aimed at me.  Damn! I find them upsetting when they are aimed at others.

This problem with hostility in Oracle forums is well recognised and I have to say I have never seen it as bad in any other IT area.  In fact the Unix ones I've used have people falling over themselves to give the best piece of advice, they don't often chuck in an RTFM because they are dying to let you know how much they know and if nothing else at least point you to a part of a document, white paper or other web site.

Tom himself talked about some of the Oracle forum issues back in 2005.

But here's a very recent question in the forums where the first answer is a sarcastic unhelpful response.

http://www.lmgtfy.com/?q=oracle+convert+long+to+varchar2
Of course the poster of the question hunted via the search engines first but you don't always manage to hit the spot.  Sometimes in google I'm just not asking the right question, my terminology isn't quite accurate enough to get me there but another DBA with experience of that area might have a few more terms to throw into the mix that means I start to get closer to where I need to be. 

Here's another one, with the first response:
The correct URL for doc questions is
http://tahiti.oracle.com
not
http://forums.oracle.com
Thank you
The poster points out they had looked at the documentation. Sometimes the Oracle documentation is a minefield.  There is so much of it, yes you can search it but it's back to the point about search words, if you are unsure of the ground you are treading you often don't get the search terms right.  This takes us back to Tom's advice, if you feel the need to point someone to the documentation why not give them a pointer of which part you are thinking of specifically.

And if you are still thinking I'm just exaggerating the 'hostility' part then how is this for a response:
Oracle documentation is SEARCHABLE.
Most people here can not be bothered to search.
They don't need guidance, they need corporal punishment or their wage subtracted for every question they ask here.
Or this
Almost all posters have 'I don't hate my work, and I want to maximize my income by doing nothing' inscribed in capitals on their forehead.
These sorts of attitudes and responses make me quite sad. This explains the need for respect in forums very well and the difference in creating passionate people it can make or break.

The people that I respect the most are the people that are always happy to share what they know, happy to re-explain the basics, when I ask a dumb question they don't make me feel small and stupid.  They often say "I'm glad you asked that, let's get back to basics, we all need to go there sometimes". I ask a lot of dumb questions especially at places like OUGs but time after time I have had people say to me later "I'm glad you asked that, I was thinking that too".  I know I'm not the only person that walks the earth that wasn't born knowing it all and hasn't managed to learn (or sometimes just plain remember) it all yet.

The people I know that time and time again are explaining the same questions with the patience of a saint as the more experienced of us have blonde moments and the newbies start to learn are the greats like Thomas Kyte and Jonathon Lewis.  I put Pete Finnagen in that pot too.  How about Jeff Smith or Martin Berger ...oh there is an endless list of helpful DBAs hanging out in twitter that will give you some wider search terms to see you closer to a problems end.

So all I am saying is when there are so many really great and good out there, why oh why has no one managed to silence the sarcastic, egotistical idiots that still think answering a question RTFM or LMGTFY is an acceptable or respectful way to talk to anyone.

OK, I'm off my soap box...until next time.

Read more...

How to create a stand alone page on Blogger

>> Sunday, 3 June 2012

If you want a permanent place on your blog for a 'Privacy and Cookie Policy',  an  ' About Me' page or  a 'Disclosure of Material Conections' then you need to use a page rather than a post.

Before making changes to your blog template please take a backup first and perhaps play around on a test blog if you are really unsure about what you are doing first.

Using the the new Blogger interface, this is the way to create a page:

Instead of posts, use the 'pages' link on the left hand side.  Then 'new page', 'blank page'

Now complete your page details just like it was a blog post.  Everything works just the same.   When you have finished, think about whether you want to accept comments or not. I don't on my 'policies'.

Once you have 'published' the page you will get a screen that asks you how you want to show it on your blog.  I choose 'don't show' - I'll show you what I do next - but have a play you might find the options there suit you just fine and then you are finished,  I find they mess up my template but that may not be the same for yours.  If it does you can come back to this screen and change it to don't show simply by clicking on the pages link on the left hand side.

 If you decide to show your page through a gadget on your blog (I'm about to show you how) you need the page url.  Just click on the page title from the 'pages' link and where it takes you to is your page,  copy the url at the top of your browser.

I show my pages through gadgets, my bar across the top of my blog has my 'About Me' page linked from it.  I use the link list gadget to do this.

In your layout screen either edit your link list or if you don't have that gadget then choose 'add gadget' from where you want it to go. And scroll down the gadgets to Linklist and click on the + sign. .
The 'new site url' is the url of your page.  The 'new site name' is  whatever you want people to see the title of it as on the link/bar/list e.g 'My Cookie Policy' or 'About Me'  Now click 'Add link'.   The arrows you see help you change the order of of your list.  Remember to 'save' to close the window you are in.


You might want to place a link to a page somewhere else on your blog.  You can do this with any type of link, it's the same as before, you just need the URL of whatever you want to link to.  So your page url is what you need in this case.

From your 'layout' screen, choose ' add gadget', the scroll down to 'html/java script' and click on the + sign.
The code you place in the Content box looks like this.  Just <a href="http://your/page/url">My Link Title</a> You can give it another Title if you like from the 'Title' box. Remember to save to exit.
Hope this help, if you have any questions I'll try to help and update this post with anything that isn't clear.

Remember your backups and perhaps play around with a test blog if you are really unsure about what you are doing first.

Read more...

  © Blogger template Simple n' Sweet by Ourblogtemplates.com 2009

Back to TOP