Interested to Buy Any Domain ? << Click Here >> for more details...
I have source file which contains duplicate data,my
requirement is unique data should pass to one file and
duplicate data should pass another file how?
- 7 Answers
- 24612 Views
- CTS, I also Faced
- E-Mail Answers
Answers were Sorted based on User's Feedback
Answer / dilip anand k
Its Simple!!
All you have to do is link your source to a Sort Stage.
Sort the data and generate a Key Change column.
Key Change column = ‘1’ represents that the record is
unique while Key Change Column = ‘0’ represents the
duplicates.
Put a Filter stage and filter out the data into two
different outputs based on the generated Key Change Column.
| Is This Answer Correct ? | 21 Yes | 5 No |
Answer / farzana kalluri
input output
1 T1 T2
2 4 1
2 6 2
1 7 3
3 4
4 5
3
5
5
6
7
for this
seq file---->Aggregate(key=id)---->filter---->2 targets
In aggregate use count rows...
in filter count=1 it goes to target1
if count=2 it goes to target2..
| Is This Answer Correct ? | 9 Yes | 3 No |
Answer / ramachandra rao
After source use aggregator stage and use option aggregator
type is count and count the records after that use filter in
where clause count>1 ie duplicate records go to one target
and another where clause count=1 ie unique records go to
another target.
| Is This Answer Correct ? | 3 Yes | 0 No |
Answer / sonali s
The above solution doesnt give required output. The requirement is as below:
Input:
A
B
B
C
D
D
D
Output should have 2 files as below.
File 1
A
C
File 2
B
B
D
D
D
Please provide solution for this
| Is This Answer Correct ? | 0 Yes | 0 No |
Answer / purba
Input:
A
B
B
C
D
D
D
Required output:
A
B
C
D
Solution:
Seq file----->sort stage(create key change column for the I/p key row)
O/p:
A 1
B 1
B 0
C 1
D 1
D 0
D 0
Now take filter stage to filter for key column=0 & keycol=1
We get 2 outputs:
A. B
B. D
C. D
D
| Is This Answer Correct ? | 0 Yes | 0 No |
Answer / riyazahamedmohamed
take two links using copystage, of your input file,one is your input file output, another one is for keychange column(using sort stage set the key change column to true) with filter "0" out of transformer, to the look up stage.set the lookup option to continue-reject.you will get the desired output.reject will capture unique records.output file will capture duplicate records.
| Is This Answer Correct ? | 0 Yes | 0 No |
Answer / krishna
As per my knowledge
initially soure is in sequential stage anc take aggrigator
stage and select the grouping option and select which column
you want to group then go to option command and select
column for calculation and select the which column you want
to do the operation .in column for calculation w have seen
many options and select missing count column name and give
the column name for output.and add transformer stage with in
the transformer stage add constraints .and give the two outputs
if column name=1 then 1 else 0
if column name>=2 then 1 else 0
it will work
| Is This Answer Correct ? | 0 Yes | 6 No |
i have source data like empno,enmae 11 ,aa 12 ,bb i want output like empno,ename 11 ,aa 12 ,bb 11 ,aa 12 ,bb
What is ds designer?
How to Remove Duplicate using SQL?
How do you register plug-ins?
what is the difference between lookup stage reject link and merge stage reject link in datastage Parallel jobs?
Hi, Please tell me how to solve this scenario in datastage ? Here we have 3 columns in a table TEST CODE,ENTRY DATE and BATCH The table looks like CODE ENTRYDATE BATCH 100 100716 1 100 100716 1 100 100716 1 200 122517 2 200 122517 2 302 555555 8 302 555555 8 302 555555 8 We need to create a seqno on grouping these 3 columns. The result should be like this. CODE ENTRYDATE BATCH SEQNO 100 100716 1 1 100 100716 1 2 100 100716 1 3 200 122517 2 1 200 122517 2 2 302 555555 8 1 302 555555 8 2 302 555555 8 3
1 Answers Alpharithm Technologies,
On which Dimension Table you implemented SCD Type in your Project
What is the difference between an operational datastage and a data warehouse?
What are the environmental settings for data stage,while working on parellel jobs?
what is the best stage in datastage parller jobs to use full out join and why
What is the use of hoursfromtime() function in transformer stage in datastage?
Source contains the metadata source ----- hyderabad,chennai,bangalore by using nested loop sequence select source as hyderabad in target just run target as hyderabad only not others please give me answer thanks in advance(give reply how to wrote logic using nested loop sequence)
Teradata 
