revisiting ancient history

Originally Published 2004-06-11 15:26:18

- what is the high-level technical design



Um... ok, I'll do my best. I didn't develop the channel. The channel has a form or forms on it that hit a forms processor running on the server. These are then compiled into a text file based on a customer-defined spec. I don't know anything about the guts of this. These files are then dropped into a folder designated to be processed. Under the old system, this folder was read once a day by a perl script that uploaded it to a server GM (or their SI) specified. About a year ago, they insisted that we make the upload secure, using something like SFTP. Great, just add an 's' to the script at the connect time, right?



Well, in actuality, the secure ftp protocol req'd another library. 15 levels of requisite libraries later, one of the requirements was upgrading perl from 5.6 to 5.8.x. Easy enough, except that other stuff on the box is using perl (i.e., everything), and ops told me that if I break other stuff I'd be strung up on a pole and they'd sick accounting on me with various forms of torture. They were pretty confident that bad things would happen if perl was messed with.



They also said, at the time, a year ago, that the server in question was being phased out ASAP anyway, and that anything I did to automate the process would be re-evaluated and either thrown away or migrated "shortly".



Which is a nice segue into the next topic...



- what is the current manual process that you run to forward leads



Ok, so I wrote a little command line batch file that automates the send process. Easy enough, right? Just launch an sftp session, pause for a few seconds, and feed the app the files.



Except it pretty much never works. I get an alert every day that says it failed, and I log on to the server, check the upload log, figure out what wasn't sent, retrieve those from the archive, and send them up by hand. 5 minutes of work. Last time I was in Europe, about a month went by before the upload was processed. Part of the problem was just that I was in hell. The other part was that the root password was changed and I had to hit up DanH for the new password. I hadn't been home in so long by that point that the dude thought I'd been fired. Go figure.



- what do you think needs to be done to automate the process



Well, I just logged on to look at the code, and it's so incredibly simple that there are a lot of things that could be done. I'm not a linux shell coder by training, and I put this together in two days -- the batch file creation making up only a couple of hours on the 2nd day.



The best solution would be to use the old code written in perl that never ever failed. There's like 500 lines of error checking in there, and all that needs to be done is an upgrade of perl. If I knew the other channels better, I'd probably have just done it anyway a year ago, but I have never had a clear understanding of everything that server is doing.



Unfortunately, I don't think anyone else does either, so the best bet is to move the GM channel off of that server altogether.



I considered, at one point, just downloading the files to my windows desktop at the office and writing my own tool using whatever libraries I wanted to perform the upload. I could actually still do that... it would be less than a day's work, but it's not a very pretty solution. We get power outages and have lurking IT staff in Dublin. Plus, well, it's my desktop for godsakes.



So there you have it.



:-)



Enjoy.