Just to add a little tidbit information for "why is this FPT behaviour done
that way":
In the early days of dBase / FoxBase, computers only has those 640kB of RAM,
from which only about 500 or so have been usable. And in FoxBase we had
whopping 64kB as reserved space for variables. Some of us might still
remember that time :)
With the invention of MultiUser programming we learned to GATHER/SCATTER our
tablefields to temporarily keep a copy in memory as well as revert data in
case of problems. The same approach is used internally by the dataengine: To
be able to revert interactive changes, the engine has to save a copy of the
fields. That approach works well with standard C/N/D/L fields, but what to
do with Memofields, which even at those dark ages already had a maximum of
lunatic 2 GigaWatts, err GigaBytes? You can't just copy 2 GBytes to a
MemVar (with 640k Memory!) Thus they decided to approach a different path
for editing Memos: Memofields get always edited at disklevel, in contrast to
Charfields, which get edited on a MemVar. To get Multiuser working, they
always locked the original Memoblock(s), copied the content to the end of
the memofile and edited there. If the user reverts the edit, then they just
scrap the just appended junk and release the lock on the original
memoblock(s), but if the user saves the changes, then the new blocknumber is
recorded in the DBF field-pointer, and the old memoblocks are then no more
referenced, thus leaving them in the memofile as junk or useless filler.
Thus, with every edit a FPT will always grow for the size of the original
blocks (plus the blocks to fit additional edits), and it always grows in
multiples of the allocated blocksize.
At VFP8 betatest times (as far as I remember) they tried to change that
behaviour of "always copy to a new block" to only occur if the table is used
sharable. In VFP9 this is not the case, a FPT also grows in exclusive mode.
The above revert-logic also debunks the "if the edit isn't changing the
original length, it will not grow" myth :) .
The block size is just a preallocated tablespace for your edit. It also pays
to analyze the best blocksize-setting for your data and do a SET BLOCKSIZE
TO x before creating a new FPT (you cannot change the blocksize of an
existing FPT). There's a gotcha here: Values from 1 to 32 are Multiples of
512 Bytes (which therefor results to blocksizes from 512 Bytes to 16384
Bytes), and from 33 to 32768 they are taken literally. (Default is 64
bytes). A value of 0 results in "no preallocated blocksize", the block
grows as needed, but will therefor always get appended. Blocksize 0 is used
on all Metadata files (SCX, MPX, VCX, FRX etc)
Another gotcha: ? SET("BLOCK") just tells you the setting for newly created
MemoTables, you need to use SYS(2012) to get the blocksize of the currrent
table.
BTW 1: It also is not possible to do some sort of recycling those dead
junkspaces in a FPT, since you don't know beforehand how big the needed
space would be to store your new data and find a fitting dead space.
BTW 2: Blocks are always used in consecutive sequence, there's no way to
connect several fragments spread over the FPT. Only the first blocknumber
gets stored in the DBF-to-FPT pointerfield.
That's why we have the PACK MEMO command, which just copies the referenced
block-chains to a new file.
-----Ursprüngliche Nachricht-----
Von: ProFox <profox-bounces(a)leafe.com> Im Auftrag von Christof Wollenhaupt
Gesendet: Dienstag, 26. Mai 2020 16:09
An: ProFox Mailinglist <profox(a)leafe.com>
Betreff: Re: [SPAM] Memo bloat
Hi Alan,
> If the changed data in bytes > blocksize then it will add a new block. If
you keep the changes < blocksize then it will re-use the same space. The
trouble is with a blocksize of 64, it's very likely that it will almost
always be adding blocks.
That would be very thoughtful of VFP, indeed, but it's unfortunately not
what is happening:
Set Blocksize To 64
Create Table memotest (memofield M)
Use memotest shared
Insert into memotest values ("1234567890")
Use
Dir like memotest.fpt
?
Use memotest shared
Replace memofield with "54321"
Use
Dir like memotest.fpt
?
Use memotest shared
Replace memofield with "1234567890"
Use
Dir like memotest.fpt
The code replaces the same field with a shorter string, then with a longer
string. All strings are way below the blocksize. Yet, each time the FPT file
grows by 64 bytes.
--
Christof