Petabyte

I have been working with a lot (when i mean a lot ~50++) of threads. It is necessary for our applications because we deal with a lot (~millions of records). Why is it necessary to multi-thread ? Let's say you have 1,000,000 records to process and each records is processes roughly about 5 seconds (really intense processing e.g. calling web services, writing xml, uploading via ftp). It will take you roughly 57 days, 20 hours, 53 minutes, 20 seconds to process all of the records ( 5,000,000 seconds ). Since you are dependent on other systems (e.g. like web services to process the records) you can't really ask the publisher of the service to speed things up (unless you have are willing to throw in time and effort into the service before you process your records). So, enter multi threading. Multi Threading - enables you to leverage the multiple processors (common to most enterprise server level boxes) to process records in parallel. BUT- multi threaded programs are really hard t