Wednesday, August 7, 2013

Example Data Pump Export script for Linux/Solaris

Data Pump Export script for Linux/Solaris


Data Pump is a command-line utility for importing and exporting objects like user tables and pl/sql source code from a Oracle database. It’s new since Oracle 10g, and it’s a better alternative for the “old” exp/imp utilities. However, do not use Data Pump to replace a full physical database backup with RMAN. Complete point-in-time recovery is not possible with Data Pump. Therefore, it should only be used for data migrations or in conjunction with RMAN.
In this blog post, I will show you how you can create a script to execute and schedule a full Data Pump export on Linux.
First, we need to define a directory object. This is an alias for a file system folder that we will need in the Data Pump script. Execute with user SYS as SYSDBA:
create directory expdir as '/expdir';
select * from dba_directories;

Note: make sure user “oracle” has write permissions on the file system folder used for the Data Pump export files (in this case: /expdir).
Next, we will create a database user that will be used for the export. This user will at least need the “EXP_FULL_DATABASE” role, CREATE SESSION and CREATE TABLE rights, and read/write access on the directory object we previously created.

Note: do NOT use “SYS as SYSDBA” for the export, this user should only be used when requested by Oracle support!

CREATE USER EXPORT
IDENTIFIED BY password
DEFAULT TABLESPACE USERS
TEMPORARY TABLESPACE TEMP
PROFILE "UNLIMITED PASSWORD EXPIRATION"
ACCOUNT UNLOCK;

GRANT CREATE SESSION TO EXPORT;
GRANT CREATE TABLE TO EXPORT;
ALTER USER EXPORT QUOTA UNLIMITED ON USERS;
GRANT EXP_FULL_DATABASE TO EXPORT;
GRANT READ, WRITE ON DIRECTORY EXPDIR TO EXPORT;
Now we will create the Linux shell script:
$ vi ora_expdp_full.sh 
 
 
 #!/bin/sh

# script to make full export of Oracle db using Data Pump

STARTTIME=`date`
export ORACLE_SID=oratst
export ORACLE_HOME=`cat /etc/oratab|grep ^${ORACLE_SID}:|cut -d':' -f2`
export EXPLOG=expdp_${ORACLE_SID}.log
export EXPDIR=/expdir
export PATH=$PATH:$ORACLE_HOME/bin
DATEFORMAT=`date +%Y%m%d`
STARTTIME=`date`

# Data Pump export
expdp export/password content=ALL directory=expdir 
dumpfile=expdp_`echo $ORACLE_SID`_%U_`echo $DATEFORMAT`.dmp 
filesize=2G full=Y logfile=$EXPLOG nologfile=N parallel=2

ENDTIME=`date`
SUBJECT=`hostname -s`:$ORACLE_SID:`tail -1 $EXPDIR/$EXPLOG`
echo -e "Start time:" $STARTTIME "\nEnd time:" $ENDTIME | mail -s "$SUBJECT" dba@mydomain.com
 
 
This script will create 2GB export files and dynamically append the date to them. So, in this case, the first file will be “expdp_oratst_01_20120503.dmp”, the second one “expdp_oratst_02_20120503.dmp”, and so on. Finally, a mail will be sent to the DBA with as mail subject the last line of the export log file, and as mail body the start and end time.
Note: You need to replace the ORACLE_SID and EXPDIR variables in the script by the ones suitable for your environment.
Make sure only the owner of the script has read and write access on it:
$ chmod 700 ora_expdp_full.sh
Finally, we can schedule the script with the Linux utility cron:
$ crontab -e
Add the following lines:
# Daily logical export
00 23 * * * /home/oracle/scripts/export/ora_expdp_full.sh 1>/dev/null 2>&1

expdp full backup thru scheduler

Scheduler and data pump expdb

Oracle Enterprise Edition 11.2.0.2
Linux x86_64

I would like to routinely export some or all of my database as part of my DR strategy. I have always used crontab to call my expdp script on a routine basis. Now I want to change that from using crontab to using Oracle Scheduler.

First I will review the shell script and parfile I use. Then I will use Oracle Scheduler to schedule the execution of this shell script.

This is my full export but with subtle modifications I also export specific schemas on a more frequent basis. I am using csh for this shell script.

This is expdp_full.sh

#!/bin/csh
# set the environment
setenv ORACLE_HOME /app/oracle/product/11.2.0.2
setenv ORACLE_SID grims
setenv ORACLE_BASE /app/oracle

setenv LOGFILE expdp_full.log
setenv DUMPFILE expdp_full.dmp
setenv DMPDIR /scripts/expdp

# expdp does not like files it wants
# to create lying around so I will clean# any up just in case.

# check for existing logfile and
# remove if found.
if ( -e ${DMPDIR}/${LOGFILE} ) then
rm ${DMPDIR}/${LOGFILE}
endif

# check for existing dump file and
# remove if found.if ( -e ${DMPDIR}/${DUMPFILE} ) then
rm ${DMPDIR}/${DUMPFILE}
endif

# change to the working directory
chdir ${DMPDIR}

# call expdp and use the full path to it.
/app/oracle/product/11.2.0.2/bin/expdp parfile=expdp_full.par DUMPFILE=${DUMPFILE} LOGFILE=${LOGFILE} DIRECTORY=DMPDIR

# once the expdp is complete zip it to save space.
# You can use the compress command in UNIX.
zip ${DUMPFILE}.`date +%m%d`.Z ${DUMPFILE}
# Get the log status for email.
set STATUS=`grep Job ${DMPDIR}/${LOGFILE}`
# mail results.

mailx -s "${ORACLE_SID} on `uname -n` FULL export '$STATUS'" email@company.com < ${DMPDIR}/${LOGFILE}

# delete the existing dump file. Zip does not remove it.
if ( -e ${DMPDIR}/${DUMPFILE} ) then
rm ${DMPDIR}/${DUMPFILE}
endif

# rename the log file to preserve it.
if ( -e ${DMPDIR}/${LOGFILE} ) then
mv ${DMPDIR}/${LOGFILE} ${DMPDIR}/${LOGFILE}.`date +%m%d`
endif
exit


The par files. This is both the full expdp and another with specific schemas.
expdp_full.parUSERID="/ as sysdba"
FULL=Y

expdp_users.par
USERID="/ as sysdba"SCHEMAS=SCOTT,HR,APPS

Of course you can stop here and use crontab to execute expdp_full.sh, but I am going to now use the Oracle Scheduler.

First I will create the scheduled job to run every morning at 2:05am. Once it runs successfully for a few days and I like the out put I will update the schedule to run this task once a week.

From sqlplus enter the new scheduled job.

BEGIN
dbms_scheduler.CREATE_JOB (
job_name => 'expdp_full',
job_type => 'EXECUTABLE',
job_action => '/oracle/scripts/expdp/expdp_full.sh',
start_date => to_date('08/14/2013 14:00:00','mm/dd/yyyy hh24:mi:ss'),
repeat_interval => 'FREQ=DAILY;BYHOUR=2;BYMINUTE=5;BYSECOND=0',
end_date => null,
enabled => TRUE,
comments => 'expdp full database');
END;

To update the frequency interval to once a week, Friday morning at 2:05am:

BEGIN
sys.dbms_scheduler.disable( '"SYS"."EXPDP_FULL"' );
sys.dbms_scheduler.set_attribute( name => '"SYS"."EXPDP_FULL"', attribute => 'repeat_interval', value => 'FREQ=WEEKLY;BYDAY=FRI;BYHOUR=2;BYMINUTE=5;BYSECOND=0');
sys.dbms_scheduler.enable( '"SYS"."EXPDP_FULL"' );
END;

Here are some useful queries to examine the scheduler jobs.

SQL> SELECT JOB_NAME, STATE FROM DBA_SCHEDULER_JOBS where job_name = 'EXPDP_FULL';
JOB_NAME STATE
------------------------------ ---------------
EXPDP_FULL SCHEDULED

SQL> SELECT * FROM ALL_SCHEDULER_RUNNING_JOBS;
no rows selected

The following will show the history of a specific job and whether it succeeded or not. Just because the job succeeded does not mean the script ran without errors. The next query is an example where this first query said SUCCEEDED yet the script failed to run.

SELECT to_char(log_date, 'DD-MON-YY HH24:MM:SS') TIMESTAMP, job_name,
job_class, operation, status FROM USER_SCHEDULER_JOB_LOG
WHERE job_name = 'EXPDP_FULL' ORDER BY log_date;

Show more detail about an job history. This is the output before I gave the full path in my shell script to extdp.

SELECT to_char(log_date, 'DD-MON-YY HH24:MM:SS') TIMESTAMP, job_name, status,
SUBSTR(additional_info, 1, 40) ADDITIONAL_INFO
FROM user_scheduler_job_run_details
WHERE job_name = 'EXPDP_FULL'
ORDER BY log_date;

14-AUG-13 14:02:02 EXPDP_FULL SUCCEEDED STANDARD_ERROR="expdp: Command not found

Friday, August 2, 2013

Basic Linux commands

 find all files in directory older than 90days
 syntax
 find <folder> -type f -mtime +30 -print
 example
  find  -type f -mtime +90 -print


delete all files in directory older than 90days
syntax
find <folder> -type f -mtime +30 -delete
example :
find  /u02/dump -type f -mtime +90 -delete

Count files in current directory

ls | wc -l


Free space on linux

Type df -h or df -k to list free disk space:

$ df -h
OR
$ df -k
Sample Output:
Filesystem             Size   Used  Avail Use% Mounted on
/dev/sdb1               20G   9.2G   9.6G  49% /
varrun                 393M   144k   393M   1% /var/run
varlock                393M      0   393M   0% /var/lock
procbususb             393M   123k   393M   1% /proc/bus/usb
udev                   393M   123k   393M   1% /dev
devshm                 393M      0   393M   0% /dev/shm
lrm                    393M    35M   359M   9% /lib/modules/2.6.20-15-generic/volatile
/dev/sdb5               29G   5.4G    22G  20% /media/docs
/dev/sdb3               30G   5.9G    23G  21% /media/isomp3s
/dev/sda1              8.5G   4.3G   4.3G  51% /media/xp1
/dev/sda2               12G   6.5G   5.2G  56% /media/xp2
/dev/sdc1               40G   3.1G    35G   9% /media/backup
 
  

df -help

[root@linux5 ~]# df --help

Usage: df [OPTION]... [FILE]...
Show information about the file system on which each FILE resides,
or all file systems by default.

Mandatory arguments to long options are mandatory for short options too.
  -a, --all             include dummy file systems
  -B, --block-size=SIZE use SIZE-byte blocks
  -h, --human-readable  print sizes in human readable format (e.g., 1K 234M 2G)
  -H, --si              likewise, but use powers of 1000 not 1024
  -i, --inodes          list inode information instead of block usage
  -k                    like --block-size=1K
  -l, --local           limit listing to local file systems
      --no-sync         do not invoke sync before getting usage info (default)
  -P, --portability     use the POSIX output format
      --sync            invoke sync before getting usage info
  -t, --type=TYPE       limit listing to file systems of type TYPE
  -T, --print-type      print file system type
  -x, --exclude-type=TYPE   limit listing to file systems not of type TYPE
  -v                    (ignored)
      --help     display this help and exit
      --version  output version information and exit

SIZE may be (or may be an integer optionally followed by) one of following:
kB 1000, K 1024, MB 1000*1000, M 1024*1024, and so on for G, T, P, E, Z, Y.

 du command examples

du shows how much space one ore more files or directories is using.
$ du -sh
103M
-s option summarize the space a directory is using and -h option provides "Human-readable" output.


To get the summary of disk usage of directory tree along with its subtrees in Megabytes (MB) only. Use the option “-mh” as follows. The “-m” flag counts the blocks in MB units and “-h” stands for human readable format.
[root@linux5]# du -mh /home/tecmint

40K     /home/tecmint/downloads
4.0K    /home/tecmint/.mozilla/plugins
4.0K    /home/tecmint/.mozilla/extensions
12K     /home/tecmint/.mozilla
12K     /home/tecmint/.ssh
673M    /home/tecmint/Ubuntu-12.10
674M    /home/tecmint


list all file extensions in current directory


find . -type f | awk -F'.' '{print $NF}' | sort| uniq -c | sort -g

Find a file "foo.bar" that exists somewhere in the filesystem

$ find / -name foo.bar -print


Find a file, who's name ends with .bar, within the current directory and only search 2 directories deep

$ find . -name *.bar -maxdepth 2 -print























Thursday, August 1, 2013

ORA-28368: cannot auto-create wallet


ORA-28368: cannot auto-create wallet

ORA-28368: cannot auto-create wallet
If you got this error simple create a directory named "wallet" on your $ORACLE_BASE/admin/$ORACLE_SID.


SQL> alter system set encryption key identified by manager;
alter system set encryption key identified by manager
*
ERROR at line 1:
ORA-28368: cannot auto-create wallet

[ora11g@dbms admin]$ mkdir -p $ORACLE_BASE/admin/$ORALE_SID/wallet

[ora11g@dbms dbms]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.1.0 Production on Thu Nov 5 13:30:43 2009

Copyright (c) 1982, 2009, Oracle. All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
With the Partitioning, Oracle Label Security, OLAP, Data Mining,
Oracle Database Vault and Real Application Testing options

SQL> alter system set encryption key identified by manager;

System altered.

SQL> exit

Wednesday, July 31, 2013

Finding out Last DML Activity on a Table

create a function for the DML logging 
 
create or replace function scn_to_timestamp_safe(p integer) return timestamp is
  e_too_old_scn exception;
  pragma exception_init(e_too_old_scn,-8181);
begin
  return 
     case 
       when p is not null then scn_to_timestamp(p)
       else null
     end;
exception 
  when e_too_old_scn then 
    return null;
end;
/
 
 now querying the last DML (insert,update,delete) for table and this cant say if 
there was any select query run or not.
 
 select 
  t.owner||'.'||t.table_name
 ,extractvalue( dbms_xmlgen.getXMLtype(q'[select nvl(scn_to_timestamp_safe
(max(ora_rowscn)),timestamp'0001-01-01 00:00:00') t 
from "]'||t.owner||'"."'||t.table_name||'"')
               ,'/ROWSET/ROW/T'
              ) last_dml
from
  all_tables t
where 
    t.IOT_TYPE is null
and t.TEMPORARY='N'
and t.NESTED='NO';
 
 
add owner if you want tables in particular schema at the end of sql.
 
 >>>     AND OWNER IN ('SCOTT'); 
 
 

Tuesday, July 23, 2013

Find all tables without primarykey in Database

As a  DBA we need to make sure that all the tables in your database are have their uniquesness
 so that the rows are not duplicate and we avoid the redundant data in our databases.

 below is the simple sql that can give you list of tables that don't have any primary key:


SELECT OWNER, table_name
FROM all_tables
MINUS
SELECT OWNER,table_name
FROM all_constraints
WHERE constraint_type = 'P' AND OWNER NOT IN ('sys','system');

you list out all the schemas you want to avoid looking for.

Monday, July 15, 2013

Easiest way to switch between schemas just with a click of button using sqqldeveloper

Easiest way to switch between schemas just with a click of button in sqldeveloper 3.2 or lower versions.This doesnt work with sqldev 4 or higher

General :  
The Schema Select extension for Oracle SQL Developer provides a convenient drop-down list which lets you choose the current schema in a sql worksheet. It also allows you to specify a default schema for newly opened worksheets.










Features:


  • Default schema can be specified in connection name

  • Current schema can be selected from a drop-down list

  • Works in Oracle SQL Developer versions 3.0, 2.1 and 1.5

  • Support for Oracle, MySQL and MS Sqlserver databases

  • Easily extendable to support other databases

This extension was created using Oracle JDeveloper and the IDE Extension SDK.

download it from :

http://javaforge.com/project/schemasel

direct download   >>>
http://javaforge.com/displayDocument/oracle.sqldeveloper.thirdparty.schemaselect.jar?doc_id=80273



Tuesday, June 25, 2013

adding primary key to already existing table in oracle

lets assume that there was a table ABC that was already existing in the database and you want to add an additional column with unique primary key values.


sql to create table abc :

  CREATE TABLE "ABC"
   (    "USERNAME" VARCHAR2(30 BYTE) NOT NULL ENABLE,
    "USER_ID" NUMBER NOT NULL ENABLE,
    "CREATED" DATE NOT NULL ENABLE )
   TABLESPACE "QUIKPAY_USER" ;




now we  can add an additional column ID which will be populated with all unique values.

alter table abc add(ID NUMBER);

you can create a sequence and get the values from the seq and insert them into table ID column:

CREATE SEQUENCE SEQ_ID
START WITH 1
INCREMENT BY 1
MAXVALUE 999999
MINVALUE 1
NOCYCLE;


now insert the unique values into the database with below sql

UPDATE abc SET ID = SEQ_ID.NEXTVAL;


now you can make the column unique or add primary key to table,so that it wont take any more duplicate value into the table.

alter table abc add primarykey (ID);