Koen,
This is a question of theory. I'm not going to lay out the entire set of procedures that get run here. Presumptuous of you to make that "unacceptable" statement without knowing what actually happens behind the scenes. I could be talking about a process that does 20 different things, checking this, that, and another thing, writing here, writing there. As another mentioned, the bulk of time was the API (outside of my actual process), anyway.
On 8/1/2020 3:30 PM, Koen Piller wrote:
Also missing the code for your procedure ProcessRecord()
Op za 1 aug. 2020 om 20:05 schreef Koen Piller koen.piller@gmail.com:
Hi,
64000 records in 17 hours???? That is way unacceptable. To run the code you published would take normally around minutes max. I suppose the most of the time is not by your code but the API call to get the data into the cursor ListOfRecsToProces, advise you to implement Codereference to find the time consuming bottle-neck. Instead of scan..endscan you may also try to make use of the SQL Update() function, this should be faster.
Koen
Op za 1 aug. 2020 om 19:07 schreef MB Software Solutions, LLC < mbsoftwaresolutions@mbsoftwaresolutions.com>:
I've got a regular process that runs, basically using key information to grab data from an API and then update the local VFP database. There are maybe 64000 records to process, and each record to update through this process takes about a second, so to process this group would take over 17 hours. Each record could be processed on its own; there are no relationships between each.
I don't want to start it and run the 64000 in a row for 17+ hours. I would like to design the app to use the table, RLOCK the row I'm processing, and the UNLOCK the row when I'm done. I figure with this design, I could run multiple instances of the MyProgram.exe (similar to how WestWind Web Connection allows you to run multiple instances) to process the batch maybe 4x faster (if I launched 4 instances of MyProgram.exe). The basic construct would be as follows:
USE ListOfRecsToProcess IN 0 SHARED Alias MyList && record is PK (to process) i, tProcessed t, tError t, cSession c(10) SCAN FOR EMPTY(cSession) AND RLOCK('MyList') IF ProcessRecord(MyList.ID) THEN REPLACE tProcessed WITH DATETIME(), cSession WITH this.cSession IN MyList ELSE REPLACE tError WITH DATETIME(), cSession WITH this.cSession IN MyList ENDIF ENDSCAN
Does anybody see any problems with that general design? The ProcessRecord method calls an API to get values and then updates the local VFP record accordingly.
tia, --Mike
-- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
[excessive quoting removed by server]