How do we eliminate duplicate records in a flat file without using Sorter and Aggregator?

Answer Posted / isha

Select all source rows.
The Dynamic Lookup transformation builds the caches from the target table.
When the lookup evaluates a row from the source that does not exist in the lookup cache, it inserts the row into the cache and assigns the NewLookupRow output port the value of 1. When the lookup evaluates a row from the source that exists in the lookup cache, it does not insert the row into cache and assigns the NewLookupRow output port the value of 0.
The filter in this mapping checks if the row is a duplicate or not by evaluating the NewLookupRow output port from the Lookup. If the value of the port is 0, the row is filtered out, as it is a duplicate row. If the value of the port is not equal to 0, then the row is passed out to the target table.

Is This Answer Correct ?    1 Yes 0 No



Post New Answer       View All Answers


Please Help Members By Posting Answers For Below Questions

Can we create a Integration service without a Repository service and vice versa?

835


What the difference is between a database, a data warehouse and a data mart?

685


source : xml file target: xml file how can we check data loaded into target xml file using writing sql query. pls help on this asap.

2383


can any one give some examples for pre sql and post sql(Except dropping and creating index).

3618


What is InformaticaPowerCenter?

786






hi friends ... i have insurance project(informatica),can you please tell me how many table and mappings we need to use?and explain me.. thank you

9288


Enlist few areas or real-time situations where informatica is required.

690


can anyone suggest best free Talend data integration training online

1348


Explain about cumulative Sum or moving sum?

703


list out all the transformations which use cache?

772


Explain what are the different types of transformation available in informatica.

664


What's the layout of parameter file (what does a parameter file contain?)?

2034


What is the session task and command task?

689


What is sq transformation?

764


what are the deliverables?in your project?

1769