What is the difference between lookup and sparse lookup?
Normal lookup will provide data for an in-memory look up
whereas Sparse lookup will access the database directly.
Normal might provide poor performance when the reference
data is huge as it has to load large data into memory.
In such scenarios a join stage may work better(It will put
the data onto an internal dataset file for each link,
and then do the join based on the data provided). If your
stream/source data is small then a sparse lookup is
suggested
as a row is retrieved from the database for each input row
instead of the full reference data being brought into
memory.
Sparse lookup sends individual SQL statement for every
incoming row (If stream/source data is huge you can imagine
the number
of times it has to hit DB and hence the down side on
performance). It can be used when you want to get the next
sequence
number from your database (Again expensive overhead on your
job as noted before). Also note that sparse lookup is only
available for DB2 and Oracle. Normal Lookup stage can have
multiple reference link but Sparse can only have one
reference
link. When normal is used it is a good practice to choose
less volume data as reference data.
| Is This Answer Correct ? | 15 Yes | 1 No |
how to find out number of records imported into source file?
Instead of using shared container in a job, I use jobs which perform similar function as Container in the sequence. Then what is the need of Shared Container?
if ename='subbu' while running job the job should be abort how come?
What are the different types of lookups in datastage?
how to call routines in stages?
Difference between ‘validated ok’ and ‘compiled’ in data stage?
Hi guys, In sequencer job, we have 3 sources, in that 1st source have some records, Here requirement is 1st source records are 100 then only run the job otherwise total job will abort... How to calculate this. please design the job. Thanks.
What are the processing stages?
Hi All, I have a file. i need to fetch the records between first and last records by using transform stage. EX:- Source: EMPNO EMPNAME 4567 shree 6999 Ram 3265 Venkat 2655 Abhi 3665 Vamsi 5852 Amit 3256 Sagar 3265 Vishnu Target: EMPNO EMPNAME 6999 Ram 3265 Venkat 2655 Abhi 3665 Vamsi 5852 Amit 3256 Sagar I dont wan't to Shree and vishnu records.we can fetch another way also but How can I write the function in transform stage?
3) Sequential file contains data like Empno ename sal 111 abc 2000 Trgt file: Trgt1----111 Trgt2----abc Trgt3---2000
i have input like this Column 1 ,column 2 3,a 4,b 5.c i want output aaa bbbb ccccc Ple help any one?
There are two file are there .1st file contains 5 records and 2nd file contain 10 records in target they want 50 records.how can achieve this