Back in the olden days I believe there was a tool called PhdBase which enabled quick or quick-ish searching of text data in memo fields. This product appears to not be available any more. Can anyone recommend an alternative? Thx
-- Alan Bourke alanpbourke (at) fastmail (dot) fm
--- StripMime Report -- processed MIME parts --- multipart/alternative text/plain (text body -- kept) text/html ---
I found FoxWeb's solution so I'll have a look at that.
I remember using that product for a search of work order text. That product is known today as TrackIT but was yanked out of FPW in alpha testing because the installer and application couldn't fit on a single floppy disk when encrypted.
Crud that was 25 years ago. I bet if you found that fll it would still work.
On Wed, Aug 1, 2018 at 7:36 AM Alan Bourke alanpbourke@fastmail.fm wrote:
I found FoxWeb's solution so I'll have a look at that.
-- Alan Bourke alanpbourke (at) fastmail (dot) fm
[excessive quoting removed by server]
Alan, I posted a few messages about it some time ago (few years!) and I did manager to track down the author Jim Korenthal Who did say he would be amenable to releasing the product as open source but I never heard anything more from him unfortunately.
I used it with VFP9 without any problems single user but it fell over when used in a multi-user environment.
Bear in mind there isn't much of its functionality you can't do (if not everything) using M$ SQL Express if you tie in the text search options. I would certainly go this way for onward support, unless of course you track down Jim!
Dave
-----Original Message----- From: ProFox profox-bounces@leafe.com On Behalf Of Alan Bourke Sent: 01 August 2018 11:31 To: profoxtech@leafe.com Subject: Optimised search of memo data.
Back in the olden days I believe there was a tool called PhdBase which enabled quick or quick-ish searching of text data in memo fields. This product appears to not be available any more. Can anyone recommend an alternative? Thx
-- Alan Bourke alanpbourke (at) fastmail (dot) fm
--- StripMime Report -- processed MIME parts --- multipart/alternative text/plain (text body -- kept) text/html ---
[excessive quoting removed by server]
Bear in mind there isn't much of its functionality you can't do (if not everything) using M$ SQL Express if you tie in the text search options. I would certainly go this way for onward support, unless of course you track down Jim!
Thanks.
SQL server select from a varchar or text data type is easy and fast.
I have 9 years of EDI inbound orders and outbound invoices that I can pound through in no time at all. When I limit the search to the past few weeks it is under a second to find a PO#
select * from EDIReceive where edistring like '%OILWAR004%' and dtmAdded >='06-01-2018'
The string could be 4000-16000 characters long depending on how many POs are in a single envelope.
On Wed, Aug 1, 2018 at 9:46 AM Dave Crozier DCrozier@flexipol.co.uk wrote:
Alan, I posted a few messages about it some time ago (few years!) and I did manager to track down the author Jim Korenthal Who did say he would be amenable to releasing the product as open source but I never heard anything more from him unfortunately.
I used it with VFP9 without any problems single user but it fell over when used in a multi-user environment.
Bear in mind there isn't much of its functionality you can't do (if not everything) using M$ SQL Express if you tie in the text search options. I would certainly go this way for onward support, unless of course you track down Jim!
Dave
-----Original Message----- From: ProFox profox-bounces@leafe.com On Behalf Of Alan Bourke Sent: 01 August 2018 11:31 To: profoxtech@leafe.com Subject: Optimised search of memo data.
Back in the olden days I believe there was a tool called PhdBase which enabled quick or quick-ish searching of text data in memo fields. This product appears to not be available any more. Can anyone recommend an alternative? Thx
-- Alan Bourke alanpbourke (at) fastmail (dot) fm
--- StripMime Report -- processed MIME parts --- multipart/alternative text/plain (text body -- kept) text/html
[excessive quoting removed by server]
I was considering a simple MSSQL table with an nvharchar(max) field to hold the VFP memo contents, and full-text indexing on it. Then it is just a question of querying that from VFP, and refreshing the records in it from VFP on some sort of timed basis.
It would seem that Jim has passed away according to my searches. I know he had movd to New York prior to my last communication and my searches gave up this:
Dave Crozier Software Development Manager Flexipol Packaging Ltd.
-----Original Message----- From: ProFox profox-bounces@leafe.com On Behalf Of Alan Bourke Sent: 01 August 2018 16:05 To: profoxtech@leafe.com Subject: Re: Optimised search of memo data.
I was considering a simple MSSQL table with an nvharchar(max) field to hold the VFP memo contents, and full-text indexing on it. Then it is just a question of querying that from VFP, and refreshing the records in it from VFP on some sort of timed basis.
Oops... never sent a link:
http://www.tributes.com/obituary/show/James-Korenthal-91427622
His age would seem to be correct but it may well be someone else, in which case apologies.
Dave Crozier Software Development Manager Flexipol Packaging Ltd.
-----Original Message----- From: ProFox profox-bounces@leafe.com On Behalf Of Alan Bourke Sent: 01 August 2018 16:05 To: profoxtech@leafe.com Subject: Re: Optimised search of memo data.
I was considering a simple MSSQL table with an nvharchar(max) field to hold the VFP memo contents, and full-text indexing on it. Then it is just a question of querying that from VFP, and refreshing the records in it from VFP on some sort of timed basis.
Does the quantity of text or the number of records or searches justify all the additional code and processes?
VFP is wickedly fast at text manipulation and searching. searching for "Rabbit" in the full text (available from Project Gutenberg) returns 101 records out of 101 in 0.03 seconds on an old and small machine.
CREATE DATABASE textsearch CREATE TABLE textsearch (iKey int autoinc, mText M, tUpdated T DEFAULT DATETIME()) FOR I=1 TO 101 INSERT INTO textsearch (mText) VALUES (textfile) NEXT SELECT * FROM textsearch WHERE "Rabbit" $ mText * returned 101 records in 0.03 seconds from a 15 Mb fpt
FOR I=1 TO 1000 INSERT INTO textsearch (mText) VALUES (textfile) NEXT SELECT * FROM textsearch WHERE "Rabbit" $ mText * returned 1101 records in 0.33 seconds from a 150 Mb FPT * quit VFP and restart for cache clearing: 0.63 seconds
On Wed, Aug 1, 2018 at 11:05 AM, Alan Bourke alanpbourke@fastmail.fm wrote:
I was considering a simple MSSQL table with an nvharchar(max) field to
hold the VFP memo contents, and full-text indexing on it. Then it is just a question of querying that from VFP, and refreshing the records in it from VFP on some sort of timed basis.
-- Alan Bourke alanpbourke (at) fastmail (dot) fm
[excessive quoting removed by server]
Ted, PHDbase went much further than straight text searches as you could match words within say 3 words of your target as well as many other options such as totally fuzzy matching. I used it successfully in a job recruitment piece of software to analyse CV's from prospective job seekers and it was blazingly fast even on Pentium II's which were new technology at the time and when I first contacted Jim. Funnily enough this thread lead me to search for Jim again and I discovered my post in FoxWikki where I remember I got Jim's email address....
http://fox.wikis.com/wc.dll?Wiki~JimKorenthal
As per Steven Black my email was/is being returned...
Dave
Dave Crozier Software Development Manager Flexipol Packaging Ltd.
-----Original Message----- From: ProFox profox-bounces@leafe.com On Behalf Of Ted Roche Sent: 01 August 2018 16:36 To: profox@leafe.com Subject: Re: Optimised search of memo data.
Does the quantity of text or the number of records or searches justify all the additional code and processes?
VFP is wickedly fast at text manipulation and searching. searching for "Rabbit" in the full text (available from Project Gutenberg) returns 101 records out of 101 in 0.03 seconds on an old and small machine.
CREATE DATABASE textsearch CREATE TABLE textsearch (iKey int autoinc, mText M, tUpdated T DEFAULT DATETIME()) FOR I=1 TO 101 INSERT INTO textsearch (mText) VALUES (textfile) NEXT SELECT * FROM textsearch WHERE "Rabbit" $ mText * returned 101 records in 0.03 seconds from a 15 Mb fpt
FOR I=1 TO 1000 INSERT INTO textsearch (mText) VALUES (textfile) NEXT SELECT * FROM textsearch WHERE "Rabbit" $ mText * returned 1101 records in 0.33 seconds from a 150 Mb FPT * quit VFP and restart for cache clearing: 0.63 seconds
On Wed, Aug 1, 2018 at 11:05 AM, Alan Bourke alanpbourke@fastmail.fm wrote:
I was considering a simple MSSQL table with an nvharchar(max) field to
hold the VFP memo contents, and full-text indexing on it. Then it is just a question of querying that from VFP, and refreshing the records in it from VFP on some sort of timed basis.
-- Alan Bourke alanpbourke (at) fastmail (dot) fm
[excessive quoting removed by server]
On Wed, 1 Aug 2018, at 4:36 PM, Ted Roche wrote:
Does the quantity of text or the number of records or searches justify all the additional code and processes?
Ted
Probably up to 100k records, each with a memo field ranging from a few words to about 4k of text, and in a multiuser context. It has to integrate into existing search functionality where users would be used to near-instant results.
Alan, My further searchings reveal that the Foxweb Full Text search facility may be of use to you:
http://www.foxweb.com/fwFullText/
No idea if it is still supported though.
Dave Crozier Software Development Manager Flexipol Packaging Ltd.
-----Original Message----- From: ProFox profox-bounces@leafe.com On Behalf Of Alan Bourke Sent: 01 August 2018 16:05 To: profoxtech@leafe.com Subject: Re: Optimised search of memo data.
I was considering a simple MSSQL table with an nvharchar(max) field to hold the VFP memo contents, and full-text indexing on it. Then it is just a question of querying that from VFP, and refreshing the records in it from VFP on some sort of timed basis.
-
Alan Bourke -
Dave Crozier -
Stephen Russell -
Ted Roche