Python saves the day!

What was a rather dull and boring day turned into an interesting one thanks to Python!

I was basking in my holiday mood as I prepare to leave London after 1.5 years of highly satisfying development project. One of my team mates is doing a small project that involves taking a file, extracting the fields and creating a HTML report out of it. It was decided to use a transformation tool that we are using in another part of our project. This tool claims to be able to transform any input schema to any output schema. Our input schema is a raw flat file which looks like:

header
record
footer
header
record
record
...
footer


Pretty simple file - but we realized that the transformer tool is not able to handle this. I cannot discuss the specifics of the tool or the file, but it is a file that confirms to a widely used standard format. While we engaged the technical support to solve this, my manager started exploring alternatives. I looked at the file and said we could do it in Python using some basic regular expressions and string operations.

He let me give it a try, and here is what I did.

1. read the file and use regex to identify the kind of record represented by the line we read
2. extract the fields in each line using the simple string[x:y] operation
3. add these fields to a dict
4. create a list of each type of record and put the dict on that list
5. then using Cheetah, render a HTML page that can show this

Because I used the lists to hold the dicts, most of my operations in the template was looping over the list and printing a values from the dict. The data was part of the header records was put in global variables and passed to the template.

At the end of the day, in about 200 lines (thanks to me putting all that coding standards and OOPS into the code) we had a credible Plan B that worked. Also we could tweak this is a few minutes to work with another file that we needed to use!

So now in case we are left in the lurch by the technical support team, we have something that will work!