What is the difference between lookup and sparse lookup?
- 1 Answers
- 6084 Views
- I also Faced
- E-Mail Answers
Normal lookup will provide data for an in-memory look up
whereas Sparse lookup will access the database directly.
Normal might provide poor performance when the reference
data is huge as it has to load large data into memory.
In such scenarios a join stage may work better(It will put
the data onto an internal dataset file for each link,
and then do the join based on the data provided). If your
stream/source data is small then a sparse lookup is
suggested
as a row is retrieved from the database for each input row
instead of the full reference data being brought into
memory.
Sparse lookup sends individual SQL statement for every
incoming row (If stream/source data is huge you can imagine
the number
of times it has to hit DB and hence the down side on
performance). It can be used when you want to get the next
sequence
number from your database (Again expensive overhead on your
job as noted before). Also note that sparse lookup is only
available for DB2 and Oracle. Normal Lookup stage can have
multiple reference link but Sparse can only have one
reference
link. When normal is used it is a good practice to choose
less volume data as reference data.
| Is This Answer Correct ? | 15 Yes | 1 No |
How many jobs in ur project? Explain any complex job u have done in ur project?
Hi , Can anyone give few examples of scenarios and there corresponding design in datastage..i am new to this tool...confused in design while my manager asking to design the job.. Please post the URL if there..so i can go through it.. Thanks in advance...
What is ibm datastage?
How much data u can get every day? 2)which data ur project contains? 3) what is the source in ur project?what is the biggest table & size in ur schema or in ur project?
What is the flow of loading data into fact & dimensional tables?
how to achieve this output ? Two Input columns(ID & Name) - ID | Name 1 | Jack 1 | Kara In output there should be only 1 column which will be populated as - 1,Jack 1,Kara
how can i get 2nd highest salary in datastage?can u send me ,thanQ 2)if i had source has 2 records 1st record ie 1st column contains 1,2,3 and 2nd coulmn contains 10,10,10 i have to get target as 2nd columns as 20,30,40 how can i?
why we use parameters instead of hard code in datastage
what is the difference between the active datawarehouse and datawarehouse
what is Audit table?Have u use audit table in ur project?
If there is a file that contains 1000 records, I need the ouput to contain these 1000 records with the header as file name concatenated with the current timestamp and trailer as the count of records
How many types of views are there in a datastage director?
Teradata 
