Fun with text files

[JaredBusch]

I have a 2GB text file with about 2.3 million records It is updated weekly.
The data is in a 200 character wide fixed width format.

In order to do what needs done, it is being brought into SQL with a BULK INSERT.
That step takes just under a minute on my machine. Not aware of any better method for this step.

The next thing it does is execute a stored proc that inserts each line into a new table with everything substringed out into pieces. that only taker another 40 seconds or so.

Then do a ton of recursive querying to link things together under certain criteria. Only 2 minutes per day there.

Related: looking at code your wrote 6 years ago is sometimes scary.

[dafyre]

@JaredBusch said in Fun with text files:

Related: looking at code your wrote 6 years ago is sometimes scary.

Especially when it works and you're not exactly sure why! lol.

[Mike Davis]

at "200 character width" I started thinking about the COBOL class I took in college.