Combining multiple execution files into one file

Combining multiple execution files into one file - using egrep in a CMD terminal

These instructions are to illustrate one method that can be used to combine all the individual daily execution files for a certain time period into one CSV file for that same period.

Step 1: From the FTP where they were generated, copy all the files you are going to combine into a separate temporary local directory. (Example : C:\Users\Brent\Desktop\Temp )

Step 2: From the Windows Start, open a command terminal (cmd.exe).

Step 3: Go to directory where the executions were saved. ( Example: cd C:\Users\Brent\Desktop\Temp )

Step 4: Run an egrep command to extract all lines in all executions files (except for header) and output to a new CSV file. You will need to have a utility like CYGWIN installed in order to run the Linux egrep command.

Example command : egrep -hv "Date/Time" EXECUTIONS* > All_Aug_Execs.csv

Command portion	Explanation:
egrep	Search file(s) for lines that match a given pattern (or non-matching lines in this case due to use of the -v option)
-hv	These two options will : suppress the filenames (-h option) in output file, and invert the match of the string entered (-v option).
"Date/Time"	Since we are searching for all inverted or "non-matching" lines of our string, we use this Date/Time string to prevent the file header from being added to our output file multiple times. (This header is seen at the beginning of each execution file we are searching.) The result of the new file should be all lines in each file EXCEPT for the header rows.
EXECUTIONS*	All files in this directory that begin with EXECUTIONS will be searched.
> All_Aug_Execs.csv	The > symbol tells to output the results in a following filename.

Step 5: The output file should now be available in your temp directory. The last thing you need to do is to add the header row to this file. Copy it from one of the daily execution files, and save it as the 1st row of your new file.