Answer / vinod upputuri

In order to collect the duplicate values:

first cal the count output col in aggregator stage
group by col.
aggregator type: count rows.
count output col..

next, use the filter stage to separate the multiple occurrence.

finally, use the join stage or lookup stage to map the two
tables join type INNER ..

then u can get the desired output..

Answer / chandu

use aggregator and calculate count of source column after
that use filter or transaformer stage use condition as count
>1 it gives only duplicate records


Thanks
Chandu-9538627859

Answer / prasad

>Agg--->Filter1------->|
| |
| |
file-->cp-------------------->Join---->Filter2---->target1
|
|
Target2
Agg: use aggregator and select Agg_type=count rows and then give the Count O/P column=Count (User defined)

Count
------------
100--2
200--1
300--3
400--1
500--1
600--1
it will generate in Agg stage then

Filter1: give condition like Count=1( u will get unique records from Filter1)

Join Stage: take Left Outer Join

Filter2:
where=column_name=''(null){u will get duplicates records)

Target1 o/p:
100
100
300
300
300

where= column_name<>''(u will get unique records)

Target2 o/p:

200
400
500
600

Please correct, if am wrong :)

Answer / sudheer

- aggregator -
seq. file - copy join - filter - seq.op


in arrg - cnt rows
in join - left outer join - left as seq.file data
in filter - where cond. - cnt>1

Answer / reddyvaraprasad

Job Design:

|----->Agg--->Filter1-->|
| |
| |
file-->cp-------------------->Join---->Filter2---->target

Agg: use aggregator and select Agg_type=count rows and then give the Count O/P column=Count (User defined).

Filter1: give the condition Count<>1

Join: select left outer join

Filter2: give the condition Count<>0

u will get the right output....what ever the duplicate records.

and if u want unique records, give the condition Count=0

Answer / reddymkl.dwh

Job Design:

Agg--->Filter1---------->|
| | Unique
file-->cp-------------------->Join---->Filter2---->target1
|
|-->Duplicate
Target2

Agg: use aggregator and select Agg_type=count rows and then give the Count O/P column=Cnt (User defined).

Filter1: give the condition Where=Cnt=1

U will get unique values like 200,400,500,600

Use Join (Or) Lookup stage: select left outer join

Filter2:

Where=Column_name='' (Duplicate values like 100,100,300,300,300)
Where=Column_name<>'' (Unique Values like 200,400,500,600)


u will get the right output....what ever the duplicate records.

Plz correct me if am wrong.....

Answer / pooja

Follow the following steps -

1. Seq file stage - Read the input data in seq file - input1.txt
2. Aggregate stage - count the number of rows (say CountRow) for each ID(group=ID)
3. Filter stage - Filter the data where CountRow<>1
4. Perform join on the output of the step 3 and input1.txt.
You will get the result :)

Answer / me

seq----> copy

from copy stage one link to aggregator apply count rows option ---> filter (on count rows output 1 ) send as reference to look up below
from copy stage second link to lookup

apply filter

WHAT ARE THE COMMON ERRORS IN DATA STAGE?

6 Answers   IBM, ME,

what is difference between migration project and integration project? can anyone explain with the example Thanks in advance

2 Answers   Satyam,

How do u view log file for a particular job?

6 Answers   HP,

why dataset ?

2 Answers  

how to change left and right links in join stage?

4 Answers   Cap Gemini, IBM,

convert yyyy mm dd to dd mm yyyy?

4 Answers  

How can we perform the 2nd time extraction of client database without accepting the data which is already loaded in first time extraction?

0 Answers   Reliance,

How to enter a log in auditing table whenever a job get finished?

2 Answers   L&T,

4) source target c1 c1 c2 c3 c2 c4 c4 c5 c3 c6 c7 c4 c5 c6 c7 please send me answer this question my mail

1 Answers  

Differentiate between hash file and sequential file?

0 Answers  

Can you highlight the main features of ibm infosphere information server?

0 Answers  

What is container and then types?

1 Answers