How to read multiple files at different directories using a
single Sequential File stage in DS Enterprise Edition

Answer Posted / venkata ramana

To read given number of files, one can use follow below

In the sequential file stage, Use
Read Method =File Pattern
File Pattern=`ls /* | head -<Number of files to be read>`

For example, I had 5 files, with similar pattern (as
date*.txt), in working folder say "/data/working/". If we
want to read three files out of these five files, one can
use following command:
`ls /data/working/date*.txt | head -3`

Just modified Venkat's Answer based on tested job.

Is This Answer Correct ?    0 Yes 0 No



Post New Answer       View All Answers


Please Help Members By Posting Answers For Below Questions

Hi All , in PX Job I have passed 4 Parameters and when i run the same job in sequence i dont want to use those parameters , is this possible if yes then how

1375


What are iconv and oconv?

1017


What are some prerequisites for datastage?

871


Difference between server jobs & parallel jobs?

881


what is the use of skid in reporting?

2163


Define orabulk and bcp stages?

922


What are some different alternative commands associated with "dsjob"?

916


What is the difference between passive stage and active stage?

1121


What are stage variables?

922


Define Job control?

866


What is apt_config in datastage?

939


1)what is the size of Fact table and dimension table? 2)how to find the size of Fact table and dimension table? 3)how to implement the surrogate key in transform stage? 4)write the configuration file path? 5)how many types of datasets explain? 6)diff b/w developed projects and migration projects? 7)how to delete the header and footer file of the sequencer file? 8)how can u call the parameters in DS in unix environment? 9) how much data ur getting daily ? 10)

2533


what is the custome stage in datastage? how can we impliment that one? plz tell me

2167


Name the third party tools that can be used in datastage?

863


In Informatica,for the table I can find coreesponding dependent mappings.Likewise can I find the dependent jobs with all the information by using the table name

2250