Answer / raj

The only solution I can get readily is using stage variables in a transformer. In order to achieve this I need to define 3 stage variables to map them to input columns. Another 3 stage variables to store the previous value. Another 3 variables to compare the values. A total of 9 stage variables are needed.

To reduce the number of stage variables, lets us checksum stage

A checksum stage generates a unique value with the given columns as input.

We supply these three columns as input and generate unique values for each combination and then use that value for comparison in the transformer

Transformer:
Stage Variable:

Checksum -> Sv1
Sv2 -> Sv1
if Sv1=Sv2 then Sv3+1 else 1 -> Sv3

Sv3 -> SEQNO

We get the desired output

How to remove duplicates in transformer stage? in parallel mode

6 Answers   Syntel, TCS,

---

What are stage variables?

0 Answers  

---

I have a source table with column name CITY having 100 records, I want target table with column name start with 'A' and 'B',remaining columns as reject outputs. how can achieve this by data stage?please help me?????

5 Answers  

---

What are the various kinds of the hash file?

0 Answers  

---

Main Function of the Staging area in DWH ?

2 Answers  

---

What could be a data source system?

0 Answers  

---

there are indexes on a table as index1 with col1, col2 index2 with col2 index3 with col1,col2,col3. if i run a query with col1='100' which index will be used and why

1 Answers   Virtusa,

---

Differentiate between Join, Merge and Lookup stage?

0 Answers  

---

Two source files contains same meta data third file contains different data types can I funnel that file.

2 Answers  

---

What is a ds designer?

0 Answers  

---

how can we do null handling in sequential files

2 Answers   Reliance,

---

Hi everyone,I have kept a project Sales And Distribution for a pharmaceutical company.can anybody explain one complex business rule that we had in our project and how did you accomplish it using DS?

0 Answers   HCL,

---