computing
  • 0

Batch Merge CSV Without Duplicate Headers

  • 0

Hi everyone,

Some background:

– I am completely brand new to DOS batch programming
– I am trying to batch merge multiple CSV files in a single folder without having the header/column name/etc duplicated.
– Each day, a new CSV file will be added to the folder.
– This batch would be used each day to create a master CSV with all data in folder.

After spending 2+ hours desperately searching for an answer on this forum, I haven’t yet found anything that works! While users have posted the exact same question, each user seems to get a difference code. After trying all the codes, I still get duplicate headers in my master file, and sometimes duplicate records!

Here is what I’m looking for (seems repetitive from other posts, but for clarity’s sake):

data1.csv:
Name Date
Sam 8/12/11
John 7/15/11
Bob 6/15/11

data2.csv
Name Date
Billy 7/14/10
Sara 9/8/11
Joey 6/2/10

data3.csv
Name Date
Hank 3/5/09
Sally 4/28/11
Gina 1/2/11

FinalData.csv
Name Date
Sam 8/12/11
John 7/15/11
Bob 6/15/11
Billy 7/14/10
Sara 9/8/11
Joey 6/2/10
Hank 3/5/09
Sally 4/28/11
Gina 1/2/11

Any help would be GREATLY appreciated!

Thanks so much,
Kevin

Share

1 Answer

  1. This should be close, but it needs testing.

    @echo off

    REM Creating FinalData.csv with Name Date at the top if it doesn’t exist already.
    if not exist FinalData.csv echo Name Date > FinalData.csv

    REM First for loop drops the top row of data1.csv
    REM and appends the rest of them to FinalData.csv
    for /F “tokens=*” %%A in (data1.csv) do (
    more + 1 > data1mod.csv
    data1mod.csv >> FinalData.csv
    )

    for /F “tokens=*” %%A in (data2.csv) do (
    more + 1 > data2mod.csv
    data2mod.csv >> FinalData.csv
    )

    for /F “tokens=*” %%A in (data3.csv) do (
    more + 1 > data3mod.csv
    data2mod.csv >> FinalData.csv
    )
    REM Deleting the files used to drop the top row
    del /f /q data1mod.csv
    del /f /q data2mod.csv
    del /f /q data3mod.csv

    • 0