I have 2 files 1st contains duplicate records only, 2nd file contains Unique records.EX:
File1:
1 subhash 10000
1 subhash 10000
2 raju 20000
2 raju 20000
3 chandra 30000
3 chandra 30000
File2:
1 subhash 10000
5 pawan 15000
7 reddy 25000
3 chandra 30000
Output file:-- capture all the duplicates in both file with count.
1 subhash 10000 3
1 subhash 10000 3
1 subhash 10000 3
2 raju 20000 2
2 raju 20000 2
3 chandra 30000 3
3 chandra 30000 3
3 chandra 30000 3

Answers were Sorted based on User's Feedback



I have 2 files 1st contains duplicate records only, 2nd file contains Unique records.EX: File1: 1 ..

Answer / subbuchamala

File1,File2====Funnel-----Copy=======1st link AGG, 2nd link JOIN----Filter----OutputFile
1. pass the 2 files to funnel stage and then copy stage.
2. from copy stage 1st link to AGG stage, 2nd link to JOIN stage
3. In AGG stage, Group by Key column say ID, NAME take the count and JOIN based on KEY column
4. Filter on COUNT>1 send the output OutputFile
we get desired output

Is This Answer Correct ?    14 Yes 0 No

I have 2 files 1st contains duplicate records only, 2nd file contains Unique records.EX: File1: 1 ..

Answer / ankit gosain

Hi,

This problem can be solved by creating a job with following
stages:

File2 File2
| |
| |
| |
File1-----Funnel----Aggregator----Join----Filter---Tgt_File
|
|
|
File1

1. Funnel both the files (Now you have Unique & Duplicates
records).
2. Aggregate on the basis of any i/p column and mention the
calculation type = Count Rows (say o/p column row_count).
3. Join the aggregated o/p with the i/p file1,2 one the
basis of key & mention the join type = Inner Join.
4. In filter stage, mention the where clause as row_count>1.

If you have further doubt or query, catch me on
ankitgosian@gmail.com

Cheers,
Ankit :)

Is This Answer Correct ?    1 Yes 0 No

Post New Answer

More Data Stage Interview Questions

You enter values in a schema file for RCP and you also entered values in sequential file? which one will it take?

1 Answers   TIAA CREF,


Define repository tables in datastage?

0 Answers  


I have Seq file, I don't want 10, 11th(or any two records like 20, 30th records ) records in the output

6 Answers   TCS,


What are the functionalities of link collector?

0 Answers  


Hi guys, please design job for this, MY INPUT IS COMPANY,LOCATION IBM,CHENNAI IBM,HYDRABAD IBM,PUNE IBM,BANGLOORE TCS,CHENNAI TCS,MUMBAI TCS,BANGLOORE WIPRO,HYDRABAD WIPRO,CHENNAI HSBC,PUNE MY OUTPUT IS COMPANY,LOCATION,COUNT IBM,chennai,hydrabad,pune,banglore,4 TCS,chennai,mumbai,bangloore,3 WIPRO,hydrabad,chennai,2 HSBC,pune,1 Thanks

3 Answers   IBM,






Define Routines and their types?

0 Answers  


How to exclude first and last lines while reading data into a sequential file(having some 1000 records).I guess probably by using unix filter option but not sure which to use

6 Answers   IBM,


what is the main differences between Lookup, Join and Merge stages?

9 Answers   HCL,


Describe link sort?

0 Answers  


What is RCP

5 Answers   CGI, CTS,


What are iconv and oconv?

0 Answers  


Hi, My source is oracle(eno,ename,sal,commision,...), my requirement is like this, if there is a null values in commission col i want to keep it as null,and for the remaining first two characters of the value in my target. Plz help me

1 Answers  


Categories