i have two sources two flatfiles same structure
one flatfile contain 100 million recs
another flatfile contain 10 million recs
i have to connect single target(performance oriented answer)
what are the steps we have to do this scenario
Answers were Sorted based on User's Feedback
Answer / babu
Hi Teja,
The two sources are haveing same structure,so beter to
use list file,to load the data into single TARGET.
Procedure:
----------
1.Place the two source paths in a note pad and
save as it '.list' format.
2.create a maping as per your requirement in
that mapping you have to take only one source no need to
take two sources.
3. Create a session and workflow.
4.In session give a SOURCE FILE PATH
is '.list' file and SOURCE FILE TYPEA as INDIRECT.
Let me know if any thing wrong.
Thanks,
dwarak.vbr@gmail.com
| Is This Answer Correct ? | 16 Yes | 0 No |
Answer / srikanth
Hi,
Both teja and Babu Answers are correct..
But here we need to consider Performance wise which one is
better.
here we have only two flat files one is having 100 million
records and second one having 10 million records.
union transformation can supports for flat files also and it
is acting as union all(don't worry about this).
first drag the 100 million records source and then drag the
10 million records source on mapping designer.
connect these two sources to union transformation.
union transformation can merge the Data By COLUMN by COLUMN.
so it is trying to merge only 10 million records with
existing 100 million records
Then we can tune the performance here.
if we have lot of flat files(same structure) then we have to
go for file indirect.
| Is This Answer Correct ? | 1 Yes | 0 No |
Answer / sudheer
As u r using source as flat file so u can go for partitioning preferably Single-Thread or Multi-Thread partitioning.
| Is This Answer Correct ? | 0 Yes | 0 No |
Answer / tejaetl
i answered this question both the flatfiles connet to union
connect to single target
| Is This Answer Correct ? | 0 Yes | 1 No |
The structure of source file is as below: Source structure(two fields) Name, Card NUmber A, 111111111(SSN) A, 01010101(Creditcard number) A, 34343434(Debit card number) B, 55555555(Creditcard number) C, 77777777(Debit card number) Target Structure(4 fields) Name,Credit card,SSN,Debit card A,01010101,111111111, 34343434 B,55555555,, C,,,77777777 Corresponding to one name there can be maximum 3 rows and minimum zero rows. Given that I do not know which record might have a particular type of number. How can I handle above requirement with informatica transformations?
What are the Differences between connected and unconnected lookup?
In which transformation you cannot drag ports into it?
In indirect file loading, suppose we have less no.of flat files then we can enter files names manually in list file creation. If millions of files are there, how can we enter the flat file names in list file?
Explain lookup transformation is active in informatica
What is the difference between writing a joiner query in ANSI style and THETA style?
Hi ETL gurus can any one tell me with a flow how to implement SCD Type 1 and SCD Type 2 in a single mapping.For some fields SCD type has to be implemented and for some fields scd type has to be implementd..Thank in advance..please let me know in case of any concerns...
How would you join a node to the already existing domain?
How do you recover rows from a failed session
what are objects in data warehouse?
Plz can anybody send Informatica and Cognos FAQ Plz To thid ID ppradeep_dwh@ayhoo.co.in
What is a sorter transformation?