How can remove duplicates in a file using UNIX?
- 3 Answers
- 11553 Views
- Cap Gemini, I also Faced
- E-Mail Answers
Answers were Sorted based on User's Feedback
Answer / j madhava rao
$sort -u filename will sorts the data and removes the duplicates lines from the output
| Is This Answer Correct ? | 17 Yes | 1 No |
Answer / vishwababu
You can use UNIQ to remove duplicates frm a file but it can
consider only same consecutive records.
Ex: if u hve records as
1 vishwa
1 Shruti
2 Ravi
3 Naveen
1 vishwa
and if u use UNIQ cmd, then you wil get vishwa, Ravi, Naveen
and vishwa.
So perform sort before using UNIQ so tat to avoid duplicates
| Is This Answer Correct ? | 7 Yes | 7 No |
Answer / prasad
Uniq -u File_name
It will remove duplicates records.
| Is This Answer Correct ? | 0 Yes | 2 No |
table a contains 5 columns and table b contains 5 columns how can i get 10 columns at target?
1. How many People are part of your Team? 2. Explain how you create jobs or flow of project? 3. Join Stage vs Lookup vs Merge Stage 4. Summation scenario based question - How you find sum of salary for a specific employee (Explain stages and flow of job)? 5. Explain Remove duplicates stage ? Can you do sort in this stage? 6. SQL Questions - Joins - Types, Difference between Join and Union 7. Unix Questions - How you run the Job, How you list all jobs in project 8. Explain Environmental Variables? 9. SQL Scenario - If you have 3 Identical record in a Table, Ex: 1, Ram, Xyz; 1, Ram, Xyz; 1, Ram, Xyz; Delete only 2 of the records and keep only 1 using the Delete query. How you will you do this?
What is difference between 8.1 , 8.5 and 9.1 ?
What is usage analysis in datastage?
Hi frnds, my scenario is like i'm having a record 1234"1323£3434%343434^23232!1212$23232 in the above record all the special characters must be removed.how can we do it in datastage 8.0.1.can any one please ans this? thanx in advance
Hi guys, please design job for this, MY INPUT IS COMPANY,LOCATION IBM,CHENNAI IBM,HYDRABAD IBM,PUNE IBM,BANGLOORE TCS,CHENNAI TCS,MUMBAI TCS,BANGLOORE WIPRO,HYDRABAD WIPRO,CHENNAI HSBC,PUNE MY OUTPUT IS COMPANY,LOCATION,COUNT IBM,chennai,hydrabad,pune,banglore,4 TCS,chennai,mumbai,bangloore,3 WIPRO,hydrabad,chennai,2 HSBC,pune,1 Thanks
How a routine is called in datastage job?
what is the new version in Datastage ? what is the diff b/t New vesion & 7.5 version?
Differentiate between odbc and drs stage?
Why do we use exception activity in Datastage?
Define project in datastage?
1.new record it will insert but changes of natural key is not present in taget i want to update (here key is composite natural key )can any one help this to explan how to do
Teradata 
