Which way to store log in flash on mpc8xx?

[Jörn Engel]

On Mon, 19 September 2005 11:40:05 -0700, Shawn Jin wrote:
> 
> > > This must be a very common task (to store logfiles in flash), but I just can't
> > > seem to find the right way to do it.
> > 
> > Note that log files may cause a lot of trouble  when  using  a  JFFS2
> > file  system. Youmay want to addd a buffering layer, like pramfs in a
> > dedicated RAM area (SRAM ideally).
> 
> What's the trouble caused by log files when using a JFFS2? I'm not
> aware of that. Any pointers to where I should look into are much
> appreciated.

I guess this should become a FAQ item.

JFFS2 creates one or more nodes whenever sys_write is called.
Unbuffered log entries are usually quite small, so a single node is
written.  Let's assume the worst case, just for fun, writes of a
single by.

Now, each node is prepended by a struct jffs2_raw_inode of 64 bytes.
So you're causing the first problem:
1. Up to 65x the raw file data is written to flash.

Which means you waste bus bandwidth, was flash space and waste flash
lifetime.  After GC, most of those nodes should get combined to 4k
nodes (plus 64 bytes for the header).  Now, JFFS2 also requires an
in-memory structure per node, the struct jffs2_raw_node_ref, weighing
in at 16 or 24 bytes, if you have a 32bit or 64bit arch.  Those will
cost you precious memory:
2. Up to 24x the raw file data is required in DRAM.

Next problem is with GC accounting.  JFFS2 only differentiates between
valid and obsolete data.  After writing a huge logfile, all of its
nodes are still valid.  Therefore GC will never try to GC such nodes,
unless they happen to sit in an erase block with a lot of obsolete
data.  And even then, they could get copied as-is, with no combining.
3. JFFS2 GC is too stupid to detect miniature nodes and combine them.

So, your problem 1 and 2 will not go away if you wait long enough.
Renaming the log file also won't help.  The only cure would be to
*copy* it to the new location and delete the original file afterwards.
Or - alternatively - to fix JFFS2 GC and send a patch.
Or - as already proposed - by writing in larger chunks in the first
place.

If you decide to fix JFFS2 GC, I'd be happy to review the patches and
merge them if deemed worthy.

Jörn

-- 
My second remark is that our intellectual powers are rather geared to
master static relations and that our powers to visualize processes
evolving in time are relatively poorly developed.
-- Edsger W. Dijkstra