i have two sources two flatfiles same structure
one flatfile contain 100 million recs
another flatfile contain 10 million recs
i have to connect single target(performance oriented answer)
what are the steps we have to do this scenario
Answers were Sorted based on User's Feedback
Answer / babu
Hi Teja,
The two sources are haveing same structure,so beter to
use list file,to load the data into single TARGET.
Procedure:
----------
1.Place the two source paths in a note pad and
save as it '.list' format.
2.create a maping as per your requirement in
that mapping you have to take only one source no need to
take two sources.
3. Create a session and workflow.
4.In session give a SOURCE FILE PATH
is '.list' file and SOURCE FILE TYPEA as INDIRECT.
Let me know if any thing wrong.
Thanks,
dwarak.vbr@gmail.com
Is This Answer Correct ? | 16 Yes | 0 No |
Answer / srikanth
Hi,
Both teja and Babu Answers are correct..
But here we need to consider Performance wise which one is
better.
here we have only two flat files one is having 100 million
records and second one having 10 million records.
union transformation can supports for flat files also and it
is acting as union all(don't worry about this).
first drag the 100 million records source and then drag the
10 million records source on mapping designer.
connect these two sources to union transformation.
union transformation can merge the Data By COLUMN by COLUMN.
so it is trying to merge only 10 million records with
existing 100 million records
Then we can tune the performance here.
if we have lot of flat files(same structure) then we have to
go for file indirect.
Is This Answer Correct ? | 1 Yes | 0 No |
Answer / sudheer
As u r using source as flat file so u can go for partitioning preferably Single-Thread or Multi-Thread partitioning.
Is This Answer Correct ? | 0 Yes | 0 No |
Answer / tejaetl
i answered this question both the flatfiles connet to union
connect to single target
Is This Answer Correct ? | 0 Yes | 1 No |
Can we create multiple integration service on single repository?
explain any diffcult scenario that u have faced in your experience... or explain any complex maping u have developed?
what is degenerated dimension table with scenario
can any one give some examples for pre sql and post sql(Except dropping and creating index).
define galaxy repository in informatica
How to display session logs based upon particular dates. If I want to display session logs for 1 week from a particular date how can I do it without using unix.
If the records doesnt satisfy the filter condition where will it go?
,if we have single SQ in flow and we have router in which it has two group and we want to join these two groups with joiner ?is it possible?
In which scenario did u used Mapping variable?
What is an Integration service in Informatica?
1:when we develop a project what are the performance issue will raise?? KPIT 2:if a table have INDEX and CONSTRAINT why it raise the performance issue bcoz when we drop the index and disable the constraint it performed better??KPIT 3:what are unix commands frequently used in informatica??
What are session parameters ? How do you set them?