VSJ – June 2003 – Sounding Board

Council member John Ellis, FIAP muses on the plethora of software tools that we can call on and the difficulties of choosing the right one for a particular circumstance.
Recently I had to create a system that allowed multiple users requesting real-time quotes to connect to a quotes engine that only allowed single processes. An additional issue was that the quotes engine was a 16-bit application. All of this was in a Microsoft Transaction Server environment using Visual Basic 6, except the quotes engine, which was in a rather dated language. The problem was to make the quotes engine appear as if it was a 32-bit transaction component.
My first task was to make a wrapper around the quotes engine that could call it and be controllable. this was fairly easy. The quotes required an input and output file and a simple call to it, which meant that quotes could be called from VB easily, with some nice frilly features like quote process times.
The next step was to provide some sort of queue into the quotes engine and to allow the user’s quotes to be put into a pipeline via MTS components and then read by the quotes processor. Several choices were available, Microsoft Message Queue MSMQ or a database. I chose the latter and used MS Access to provide the pipeline with a table of incoming quote requests and a table for the results. These requests had unique IDs on which the MTS components could wait for a reply. The results could be retrieved when available, sleeping for half a second between checks for a reply.
All was going fine. Quotes were being queued, processed and sent back to the user in a timely manner. That was until we started load testing. Within a few minutes it was obvious we were having locking problems. Quotes could not be added or deleted from the queues as each process either queuing or retrieving quotes was locking another as soon as there were a few users. Timeouts occurred, the database threw back errors and the VB control program just kept reporting message after message, all errors.

So, back to square one. Cup of tea, a bit of hair pulling and, of course, a project manager going on and on about why does it not work, can we get a faster machine etc. I considered the MSMQ approach, but that requires SQL server and its own service to run it, which seems a bit like overkill for such a simple process.
Then an idea struck me. One of those gems I’ve never found a use for, ‘File System Object’, could be enlisted. This allowed us to interact with the file system of the PC, so instead of a database, we could use the directory structure and instead of tables we had an IN and OUT directory. Now we could write separate files into the IN box (effectively a record). The file system does not lock. In fact, the files can easily be opened, read and deleted without any locking issues at all. Altogether, we had a simpler idea, fewer overheads in terms of memory use, no database access and, on top of this, the process runs four times quicker.
The moral of this is, of course, that software developers have a vast array of tools at their beck and call. Some are very powerful and are the subject of charges (Access, SQL, MSMQ etc.), while some tools, like FSO, are free and can be used for purposes they were not necessarily designed for. Have you got other examples you’d like to share?
You can contact John at john.ellis@wellis-technology.co.uk.
[Something you’d like to get off your chest? Email me (Robin Jones) at eo@iap.org.uk.]