On Sat, May 7, 2016 at 10:48 PM, Tomasz Janeczko groups@amibroker.com [amibroker] <amibroker@yahoogroups.com> wrote:
If all you're doing is relying on the automatic buffering of <cstdio> functions, and each call to AFL fclose() results in a call to <cstdio> fclose(), then that does *not* guarantee that writing to disk is delayed until scan/exploration completion. Once you call C's fclose() (which flushes program buffers to operating system buffers), the decision about when to write the data to disk is passed on to Windows, and your code has no further control. Since we are forced to call fclose() for every symbol, this means that Windows is free to command the hard drive to write each symbol's data to disk with each iteration of the scan/exploration loop. In short, this means that this level of buffering is virtually non-existent.
Hello,
fopen/fputs/fclose are fully buffered.
Actually they are buffered twice, first buffer is in the C runtime library.
If, on the other hand, the AFL engine has its own buffer, which it only flushes at the end of the scan/exploration (or when the buffer is considered to require too much RAM), then that is a totally different story. In this case, AFL fclose() only executes a "virtual" file close. This should be documented in https://www.amibroker.com/guide/afl/fclose.html , because C programmers will assume that fclose() flushes the program buffer to the operating system buffer like <cstdio> fclose().
Second buffer is in Windows OS.
Which is a joke! Yes, I have Windows file write buffering enabled. Unlike in other operating systems, it is very easy to make a hard drive almost totally unresponsive under Windows by simply giving it two simultaneous writing tasks. So practically speaking, this level of buffering is also non-existent.
Enough hard drive crashes under Windows have taught me to take complete control of all file write buffering and scheduling. Currently, AFL's lack of "post processing" support does not allow for safe management of disk writing, except with some of the ugly hacks that have been posted.
You may feel that offering "post processing" abilities to AFL scan/exploration code would break the "multi-threading" paradigm, but I'd rather break the paradigm than break my hard drive. If you don't like the Status("lastsymbol") semantics that I proposed because it suggests the wrong paradigm, then use something else--I don't care about the semantics.
Robbie Geary
__._,_.___
Posted by: Robbie Geary <rgearyiii@gmail.com>
| Reply via web post | • | Reply to sender | • | Reply to group | • | Start a New Topic | • | Messages in this topic (49) |
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
.
__,_._,___
EmoticonEmoticon