how many types of remove the duplicate records?

Answers were Sorted based on User's Feedback



how many types of remove the duplicate records?..

Answer / prabhu rathnam

1. remove duplicate stage
2. sort stage
3. copy stage

Is This Answer Correct ?    8 Yes 0 No

how many types of remove the duplicate records?..

Answer / rohit babar

There are 3 ways by which we can remove the duplicate records
1. By using remove duplicate stage. We have control here on which record we want to remove it's first or last. There is option record to retain in remove duplicate stage where we select first or last option.

2. By using sort stage. there is option allow duplicate in sort stage. We set this option true when we want duplicate records & If we want unique records we set it to the false

3. By in-line sorting. In Partition tab of any stage when we select key base partition tech. then perform sort option will enable if we check this option next 2 option will enable which is stable & unique if we select stable that means we allow duplicate records & if we select unique that means we remove duplicate records and getting unique records

Is This Answer Correct ?    5 Yes 0 No

how many types of remove the duplicate records?..

Answer / soumya

1. Sort the Data using a key column then Use 3 transformer stage Variable
SV1 = Col1
SV2 = If SV1 = SV3 then 'DUP' else 'UNQ'
SV3 (Initial value = 0)
In the constraint SV2 = 'DUP' or SV2 = 'UNQ'

2. Aggregater stage group by key col (col1) count the column
then a filter stage use count col > 1.  

Is This Answer Correct ?    1 Yes 2 No

Post New Answer

More Data Stage Interview Questions

1)s.key generate 1 to 700 records today. tomorrow another 400 will updated how to update the records using s.key generator? 2)source is like :-- DB --> T/F stage1 --> seq1file T/f 1 is linking with T/F2 ---> seq 2 how to load the data? in source i given some conditions those r going in seq1. The another data will going to seq2 how to do this ?

0 Answers   Wipro,


I have source like "abcdefghijklmnopqrs", how to change 10th character from j to z in datastage?

2 Answers  


how do you pass parameters in a script?

1 Answers   L&T,


What is the difference between SQl Loader and OCI in datastage?

2 Answers   CSC,


how many types of sorting the data in data stage?

1 Answers   BoA, IBM,






How we can convert rows to columns in datastage?

4 Answers   IBM,


CAN WE DECLARE DATASET SET WITH OTHER THAN .DS EXTENTION OR NOT IF YES WHAT IS THE DIFFERENCE U FIND?IF NOT WHY?

2 Answers  


how many rows sorted in sort stage by default in server jobs

0 Answers   Syntel,


What is the Difference Between DataStage 7.5 version and 8.1 Version?

10 Answers   IBM,


root tree will find which is server job and which is parallel job?

0 Answers   IBM,


hi iam new to this tooliam cmpltied to know abt datastage so now iam in project tell me whole step by step what iam doing iwnt to go with exp so plz hlp me pals

0 Answers  


what are the devoleper roles in real time? plz tell i am new to datastage....

0 Answers   Mphasis,


Categories