Re: Performance of Large Arrays and QuickSort in SQR

["Wanko, Christopher G, CFCTRCFFIN" <apollo@ATT.COM>]

> Which  of these two is faster will largely depend on your
> database and operating system.  The overhead for creating a
> temporary table and then doing a sorted query is fairly
> substantial, as is creating an operating system file and
> running an operating system sort program.  In any case, I
> would expect SQR to perform the read/sort/write operation
> very quickly.  SQR would also be a more portable solution
> (across databases and operating systems) if that's an issue.


I suppose SQR would get an edge in portability if the database access was
consistent across platforms (recompile for target machines), but in the case
of any flat-file processing, I would suggest that sorting is best done by
programs designed to sort.

Anecdotal evidence: SyncSort can do pretty complex processing and sorting
jobs
in about a tenth of the time it would take an SQR program- any SQR program.
While I would hesitate to add another step to your process, I would also
hesitate
to try to use SQR as the multipurpose tool for all my needs.

You have need for very fast sorting and perhaps simple data validation.
SyncSort
would do it faster (and I'm not a rep for SyncSort, but it really does fit
your
situation), and the end result is a flatfile or flatfiles ready for loading.
This is
where your SQR program(s) would be more useful.

In general, unless you don't have access to anything else, try to get the
best tools
for the job.  In this case, flatfile processing through a program such as
SyncSort
will greatly simplify your SQR coding while improving your speed of
execution.

-Chris