Saturday, December 20, 2014

Partitioning made easy



One of the nice features in Oracle 12c is the easier mainteannace of partitioning operations. 

In pre-12c oracle databases , to add , split , merge the partitions ,  the commands has to be done per partition .  Starting with Oracle 12c , the operations can be done in one single statement.

Let us see an example.

We will create a table range based partitioned table called t_part with two partitions.




SQL> Create table t_part
 2  ( productcode  int ,  productname varchar2(150) , company varchar2(150))
 3  partition by range (productcode)
 4  (
 5  partition product_1000 values less than (1000) ,
 6  partition product_2000 values less than (2000)
 7  )
 8  ;


Table created.




To add partitions in prior to 12c , we would need to issue three statements as below.

SQL> alter table t_part
 2  add partition product_3000 values less than ( 3000) ;


Table altered.

SQL> alter table t_part
 2  add partition product_4000 values less than ( 4000) ;

Table altered.

SQL> alter table t_part
 2  add partition product_5000 values less than ( 5000) ;

Table altered.

For the similar operations in oracle 12c , we can do it in one statement as below. 
In the following example , I am adding five partitions in one statement.


SQL> alter table t_part
 2  add
 3  partition product_6000 values less than ( 6000) ,
 4  partition product_7000 values less than ( 7000) ,
 5  partition product_8000 values less than ( 8000) ,
 6  partition product_9000 values less than ( 9000),
 7  partition product_10000 values less than ( 10000),
 8  partition product_15000 values less than ( 15000)
 9  ;

Table altered.


SQL> insert into t_part
 2  select object_id as productcode
 3  , object_name as productname
 4  , owner as company from all_objects
 5  where object_id <15000 span="">
 6  ;

14602 rows created.
SQL> exec dbms_stats.gather_table_stats ( user , 'T_PART') ;
PL/SQL procedure successfully completed.

SQL> select  table_name , partition_name , num_rows ,
 2  last_analyzed
 3  from user_tab_partitions where table_name ='T_PART'
 4  order by 2 ;

TABLE_NAME PARTITION_NAME         NUM_ROWS LAST_ANAL
---------- -------------------- ---------- ---------
T_PART     PRODUCT_1000                889 20-DEC-14
T_PART     PRODUCT_10000               986 20-DEC-14
T_PART     PRODUCT_15000              4978 20-DEC-14
T_PART     PRODUCT_2000                924 20-DEC-14
T_PART     PRODUCT_3000               1000 20-DEC-14
T_PART     PRODUCT_4000               1000 20-DEC-14
T_PART     PRODUCT_5000               1000 20-DEC-14
T_PART     PRODUCT_6000                985 20-DEC-14
T_PART     PRODUCT_7000                944 20-DEC-14
T_PART     PRODUCT_8000                928 20-DEC-14
T_PART     PRODUCT_9000                968 20-DEC-14

11 rows selected.


Similarly , I can split the partition in one statement as well as shown below.





SQL> alter table t_part
 2  split
 3  partition product_15000
 4  into
 5  (
 6  partition product_11000 values less than ( 11000) ,
 7  partition product_12000 values less than ( 12000) ,
 8  partition product_13000 values less than ( 13000) ,
 9  partition product_14000 values less than ( 14000) ,
10  partition product_15000
11  ) ;


Table altered.

SQL> select  table_name , partition_name , num_rows ,
 2  last_analyzed
 3  from user_tab_partitions where table_name ='T_PART'
 4  order by 2 ;

TABLE_NAME PARTITION_NAME         NUM_ROWS LAST_ANAL
---------- -------------------- ---------- ---------
T_PART     PRODUCT_1000                889 20-DEC-14
T_PART     PRODUCT_10000               986 20-DEC-14
T_PART     PRODUCT_11000
T_PART     PRODUCT_12000
T_PART     PRODUCT_13000
T_PART     PRODUCT_14000
T_PART     PRODUCT_15000
T_PART     PRODUCT_2000                924 20-DEC-14
T_PART     PRODUCT_3000               1000 20-DEC-14
T_PART     PRODUCT_4000               1000 20-DEC-14
T_PART     PRODUCT_5000               1000 20-DEC-14

TABLE_NAME PARTITION_NAME         NUM_ROWS LAST_ANAL
---------- -------------------- ---------- ---------
T_PART     PRODUCT_6000                985 20-DEC-14
T_PART     PRODUCT_7000                944 20-DEC-14
T_PART     PRODUCT_8000                928 20-DEC-14
T_PART     PRODUCT_9000                968 20-DEC-14

15 rows selected.


This enhancement is one of the best in Oracle 12c.

Welcome your comments.

Monday, December 15, 2014

Filtered index in SQL Server / Oracle's equivalent



One of my favorite things in SQL Server is Filtered index .  Filtered index helps us in higher selectivity and also in decreased storage consumption , if the  query criteria looks only for pre defined. sub set of data . 

Let us say , you got a table of 50 million records , out of which only 50,000 are active records  and your application only looks for active records . Then filtered index will make much more sense . 

Here is an example. 



 

 


As you can see , there is a significant difference in the index size. 
The query optimizer uses the second indexes , whenever , there is query looks for any ACTIVE product. 































In Oracle  , we can simulate the filtered index with Function Based Index . 


SQL> drop table t purge ;

Table dropped.

SQL> Create table t as select * from all_objects ;

Table created.

SQL> Create index idx1_t on t( owner , object_type ) ;

Index created.

SQL> exec dbms_stats.gather_table_stats(user , 'T') ;

PL/SQL procedure successfully completed.

SQL> analyze index idx1_t validate structure ;

Index analyzed.

SQL> col name format a15
SQL> Select name , height , blocks , lf_rows , lf_blks , btree_space , used_space  from index_stats ;

NAME                HEIGHT     BLOCKS    LF_ROWS    LF_BLKS BTREE_SPACE USED_SPACE
--------------- ---------- ---------- ---------- ---------- ----------- ----------
IDX1_T                   3        384      93172        336     2712096    2415824


Now , let us create a FBI , where I am interested only in 'MATERIALIZED VIEW' of a user. 

SQL> Create index idx2_t on t( owner , case when object_type = 'MATERIALIZED VIEW' then 'MATERIALIZED VIEW' end ) ;

Index created.

SQL> analyze index idx2_t validate structure ;

Index analyzed.

SQL> Select name , height , blocks , lf_rows , lf_blks , btree_space , used_space  from index_stats ;

NAME                HEIGHT     BLOCKS    LF_ROWS    LF_BLKS BTREE_SPACE USED_SPACE
--------------- ---------- ---------- ---------- ---------- ----------- ----------
IDX2_T                   2        256      93172        232     1864032    1664870


As you can see, the storage has decreased from 2712096 to 1864032  , leaf blocks from 336 to 232 and also like the height of the index from 3 to 2 , which helps the optimizer to relatively find the data faster. 


Comments welcome.




Thursday, December 11, 2014

Minimize logging for Data Pump Import - IMPDP



Starting in Oracle 12c, we  can disable ( well ... minimize) the logging by setting the new setting in the TRANFORM parameter . Logging is not totally eliminated but it will be minimal . This setting has no effect if the database is in FORCE LOGGING mode. 

This is useful in loading large data sets or creating new databases .  With this setting , you can minimize the logging for table or index or both. 

if we look at the help at the command prompt ( impdp -help) , we can see there are few settings have added. 

11g 


TRANSFORM
Metadata transform to apply to applicable objects.
Valid keywords are: OID, PCTSPACE, SEGMENT_ATTRIBUTES and STORAGE.


12c

TRANSFORM
Metadata transform to apply to applicable objects.
Valid keywords are: DISABLE_ARCHIVE_LOGGING, LOB_STORAGE, OID, PCTSPACE,

SEGMENT_ATTRIBUTES, STORAGE, and TABLE_COMPRESSION_CLAUSE.


Here are the examples.

In the following example , the logging is disabled for both table and index.

C:\>impdp zahir schemas=zahir directory=dataad dumpfile=zdata.dmp logfile=zdata.log  remap_schema=zahir:testuser transform=disable_archive_logging:Y

Import: Release 12.1.0.1.0 - Production on Thu Dec 11 14:33:04 2014

Copyright (c) 1982, 2013, Oracle and/or its affiliates.  All rights reserved.
Password:

Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, Oracle Label Security, OLAP, Advanced Analytics
and Real Application Testing options
Master table "ZAHIR"."SYS_IMPORT_SCHEMA_01" successfully loaded/unloaded
Starting "ZAHIR"."SYS_IMPORT_SCHEMA_01":  zahir/******** schemas=zahir directory=dataad dumpfile=zdata.dmp logfile=zdata.log remap_schema=zahir:testuser transform=disable_archive_logging:Y
Processing object type SCHEMA_EXPORT/USER
ORA-31684: Object type USER:"TESTUSER" already exists
Processing object type SCHEMA_EXPORT/SYSTEM_GRANT
Processing object type SCHEMA_EXPORT/ROLE_GRANT
Processing object type SCHEMA_EXPORT/DEFAULT_ROLE
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
. . imported "TESTUSER"."ZAHIRTESTL"                     12.50 MB  110350 rows
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Processing object type SCHEMA_EXPORT/STATISTICS/MARKER
Processing object type SCHEMA_EXPORT/POST_SCHEMA/PROCACT_SCHEMA
Job "ZAHIR"."SYS_IMPORT_SCHEMA_01" completed with 1 error(s) at Thu Dec 11 14:33:18 2014 elapsed 0 00:00:11

In the following example , the logging is disabled for table. 


C:\>impdp zahir schemas=zahir directory=dataad dumpfile=zdata.dmp logfile=zdata.log  remap_schema=zahir:testuser transform=disable_archive_logging:Y:TABLE

Import: Release 12.1.0.1.0 - Production on Thu Dec 11 14:34:27 2014

Copyright (c) 1982, 2013, Oracle and/or its affiliates.  All rights reserved.
Password:

Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, Oracle Label Security, OLAP, Advanced Analytics
and Real Application Testing options
Master table "ZAHIR"."SYS_IMPORT_SCHEMA_01" successfully loaded/unloaded
Starting "ZAHIR"."SYS_IMPORT_SCHEMA_01":  zahir/******** schemas=zahir directory=dataad dumpfile=zdata.dmp logfile=zdata.log remap_schema=zahir:testuser transform=disable_archive_logging:Y:TABLE
Processing object type SCHEMA_EXPORT/USER
ORA-31684: Object type USER:"TESTUSER" already exists
Processing object type SCHEMA_EXPORT/SYSTEM_GRANT
Processing object type SCHEMA_EXPORT/ROLE_GRANT
Processing object type SCHEMA_EXPORT/DEFAULT_ROLE
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/TABLE/TABLE
ORA-39151: Table "TESTUSER"."ZAHIRTESTL" exists. All dependent metadata and data will be skipped due to table_exists_action of skip
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Processing object type SCHEMA_EXPORT/STATISTICS/MARKER
Processing object type SCHEMA_EXPORT/POST_SCHEMA/PROCACT_SCHEMA
Job "ZAHIR"."SYS_IMPORT_SCHEMA_01" completed with 2 error(s) at Thu Dec 11 14:34:31 2014 elapsed 0 00:00:01


In the following example , the logging is disabled for index.

C:\>impdp zahir schemas=zahir directory=dataad dumpfile=zdata.dmp logfile=zdata.log  remap_schema=zahir:testuser transform=disable_archive_logging:Y:INDEX

Import: Release 12.1.0.1.0 - Production on Thu Dec 11 14:34:44 2014

Copyright (c) 1982, 2013, Oracle and/or its affiliates.  All rights reserved.
Password:

Connected to: Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, Oracle Label Security, OLAP, Advanced Analytics
and Real Application Testing options
Master table "ZAHIR"."SYS_IMPORT_SCHEMA_01" successfully loaded/unloaded
Starting "ZAHIR"."SYS_IMPORT_SCHEMA_01":  zahir/******** schemas=zahir directory=dataad dumpfile=zdata.dmp logfile=zdata.log remap_schema=zahir:testuser transform=disable_archive_logging:Y:INDEX
Processing object type SCHEMA_EXPORT/USER
ORA-31684: Object type USER:"TESTUSER" already exists
Processing object type SCHEMA_EXPORT/SYSTEM_GRANT
Processing object type SCHEMA_EXPORT/ROLE_GRANT
Processing object type SCHEMA_EXPORT/DEFAULT_ROLE
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/TABLE/TABLE
ORA-39151: Table "TESTUSER"."ZAHIRTESTL" exists. All dependent metadata and data will be skipped due to table_exists_action of skip
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Processing object type SCHEMA_EXPORT/STATISTICS/MARKER
Processing object type SCHEMA_EXPORT/POST_SCHEMA/PROCACT_SCHEMA
Job "ZAHIR"."SYS_IMPORT_SCHEMA_01" completed with 2 error(s) at Thu Dec 11 14:34:47 2014 elapsed 0 00:00:01


I welcome your comments. 


Monday, October 20, 2014

RMAN Enhancements in 12c - Part 1



To execute a SQL statement in RMAN prompt , we would need to prefix  "sql" in  Oracle 11g and below . In Oracle 12c , we can issue sqls directly from RMAN prompt.   This enhancement is helpful in scripting RMAN backup jobs.

Here are the examples.

C:\>rman target /

Recovery Manager: Release 12.1.0.1.0 - Production on Mon Oct 20 11:13:56 2014

Copyright (c) 1982, 2013, Oracle and/or its affiliates.  All rights reserved.

connected to target database: TEST12DB (DBID=367699687)

RMAN> create user backupuser identified by bkup1user ;

using target database control file instead of recovery catalog
Statement processed

RMAN> alter user backupuser default tablespace users;

Statement processed

RMAN> alter user backupuser quota unlimited on users;

Statement processed


RMAN> select banner from v$version ;

using target database control file instead of recovery catalog
BANNER
--------------------------------------------------------------------------------
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
PL/SQL Release 12.1.0.1.0 - Production
CORE    12.1.0.1.0      Production
TNS for 64-bit Windows: Version 12.1.0.1.0 - Production
NLSRTL Version 12.1.0.1.0 - Production

Monday, September 22, 2014

Backport of Data Redaction in 11.2.0.4


Data Redaction has been backported to Oracle 11.2.0.4 .  See the examples below . 
For the original post on Data Redaction , please see this link http://mfzahirdba.blogspot.com/2014/04/data-redaction-in-oracle.html. 

I would not upgrade to 11.2.0.4 , as the support for this release ends soon ( in few months from now) . :-( 

Examples below 


SQL> select * from v$version ;

BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
PL/SQL Release 11.2.0.4.0 - Production
CORE    11.2.0.4.0      Production
TNS for 64-bit Windows: Version 11.2.0.4.0 - Production
NLSRTL Version 11.2.0.4.0 - Production

SQL> create table employee as
  2      select lpad(object_id * object_id, 9, 5)  ssn , object_name ename  ,
  3          object_id * 2 as empid
  4          from all_objects
  5          where object_id > 70000  and rownum <= 10 ;

Table created.

SQL> col ename format a30
SQL> Select * from employee;

SSN       ENAME                               EMPID
--------- ------------------------------ ----------
490014000 USER_XML_SCHEMAS                   140002
490042000 DBA_XML_INDEXES                    140006
490056001 ALL_XML_INDEXES                    140008
490070002 ALL_XML_INDEXES                    140010
490084003 USER_XML_INDEXES                   140012
490098004 USER_XML_INDEXES                   140014
490112006 USER_XML_COLUMN_NAMES              140016
490126008 USER_XML_COLUMN_NAMES              140018
490154012 DBA_XML_SCHEMA_IMPORTS             140022
490182016 DBA_XML_SCHEMA_INCLUDES            140026

10 rows selected.

SQL>            BEGIN
  2            DBMS_REDACT.ADD_POLICY(
  3              object_schema        => 'scott',
  4              object_name          => 'employee',
  5              column_name          => 'ssn',
  6              policy_name          => 'hide_emp_ssn',
  7              function_type        => DBMS_REDACT.FULL,
  8               function_parameters => DBMS_REDACT.REDACT_US_SSN_F5,
  9              expression           => 'SYS_CONTEXT(''USERENV'',''CLIENT_INFO'') != ''PAYROLL''');
 10        END;
 11        /

PL/SQL procedure successfully completed.

SQL> exec dbms_application_info.set_client_info('PAYROLL') ;

PL/SQL procedure successfully completed.

SQL> Select SYS_CONTEXT('USERENV','CLIENT_INFO') from dual ;

SYS_CONTEXT('USERENV','CLIENT_INFO')
--------------------------------------------------------------------------------
PAYROLL

SQL> Select * from employee;

SSN       ENAME                               EMPID
--------- ------------------------------ ----------
490014000 USER_XML_SCHEMAS                   140002
490042000 DBA_XML_INDEXES                    140006
490056001 ALL_XML_INDEXES                    140008
490070002 ALL_XML_INDEXES                    140010
490084003 USER_XML_INDEXES                   140012
490098004 USER_XML_INDEXES                   140014
490112006 USER_XML_COLUMN_NAMES              140016
490126008 USER_XML_COLUMN_NAMES              140018
490154012 DBA_XML_SCHEMA_IMPORTS             140022
490182016 DBA_XML_SCHEMA_INCLUDES            140026

10 rows selected.

SQL>  exec dbms_application_info.set_client_info('MIS') ;

PL/SQL procedure successfully completed.

SQL> Select SYS_CONTEXT('USERENV','CLIENT_INFO') from dual ;

SYS_CONTEXT('USERENV','CLIENT_INFO')
--------------------------------------------------------------------------------
MIS

SQL> Select * from employee;

SSN       ENAME                               EMPID
--------- ------------------------------ ----------
          USER_XML_SCHEMAS                   140002
          DBA_XML_INDEXES                    140006
          ALL_XML_INDEXES                    140008
          ALL_XML_INDEXES                    140010
          USER_XML_INDEXES                   140012
          USER_XML_INDEXES                   140014
          USER_XML_COLUMN_NAMES              140016
          USER_XML_COLUMN_NAMES              140018
          DBA_XML_SCHEMA_IMPORTS             140022
          DBA_XML_SCHEMA_INCLUDES            140026

10 rows selected.

SQL>

Friday, September 12, 2014

Color / Colour the SQL

We use AWR and ASH to troubleshoot performance ( provided we licensed diagostic pack option )  .
Typically , only the top consumers ( SQLs ) is displayed in AWR. 

You can color / colour the SQL to  see the execution plans and other metrics of a SQL , 
even if the SQL does not qualify as top consumer ( if the SQL is in the cache at the snapshot time) . 
The utility is ADD_COLORED_SQL  in DBMS_WORKLOAD_REPOSITORY.  

As a side note , I found a documentation bug in oracle 11g documentation . It was wrongly documented as DBMS_WORKLOAD_REPOSITORY.ASH_REPORT_HTML , instead of ADD_COLORED_SQL.  But this is corrected in Oracle 12c documentation. 




  
Here is an example. 

I am using "ZAHIR_COLOR_SQL" as comment just for clarity . You need not comment to get the SQL id. 


Session 1 :
---------

SQL> show user
USER is "SCOTT"
SQL> desc emp
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 EMPNO                                     NOT NULL NUMBER(4)
 ENAME                                              VARCHAR2(10)
 JOB                                                VARCHAR2(9)
 MGR                                                NUMBER(4)
 HIREDATE                                           DATE
 SAL                                                NUMBER(7,2)
 COMM                                               NUMBER(7,2)
 DEPTNO                                             NUMBER(2)

SQL> Select /* ZAHIR_COLOR_SQL */ empno , ename from  emp;

     EMPNO ENAME
---------- ----------
      7369 SMITH
      7499 ALLEN
      7521 WARD
      7566 JONES
      7654 MARTIN
      7698 BLAKE
      7782 CLARK
      7788 SCOTT
      7839 KING
      7844 TURNER
      7876 ADAMS

     EMPNO ENAME
---------- ----------
      7900 JAMES
      7902 FORD
      7934 MILLER

14 rows selected.


Go to another session and determine the SQL ID of this SQL and color it. 

SQL> select sql_id , sql_text  from v$sql where sql_text like '%ZAHIR_COLOR%';

SQL_ID
-------------
SQL_TEXT
--------------------------------------------------------------------------------
dfjudnqgk5c11
select sql_id , sql_text  from v$sql where sql_text like '%ZAHIR_COLOR%'

drfh6qd1ctrb7
Select /* ZAHIR_COLOR_SQL */ empno , ename from  emp


SQL> exec DBMS_WORKLOAD_REPOSITORY.ADD_COLORED_SQL(sql_id  => 'drfh6qd1ctrb7');

PL/SQL procedure successfully completed.


SQL> exec dbms_workload_repository.create_snapshot();

PL/SQL procedure successfully completed.


We can look at the execution plan using dbms_xplan or via AWR from Enterprise Manager or AWR  by executing awrrpt.sql . 

Here is an example of getting the execution plan from AWR as shown below. 


SQL> select * from table(dbms_xplan.display_awr('drfh6qd1ctrb7'))
  2  ;

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------
SQL_ID drfh6qd1ctrb7
--------------------
Select /* ZAHIR_COLOR_SQL */ empno , ename from  emp

Plan hash value: 3956160932

--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |       |       |     3 (100)|          |
|   1 |  TABLE ACCESS FULL| EMP  |    14 |   140 |     3   (0)| 00:00:01 |
--------------------------------------------------------------------------


13 rows selected.



Comments Welcome.