Stephen Foskett, Pack Rat - Latest Comments in We Need a Storage Revolution

Re: We Need a Storage Revolution

Caitlin Bestler — Thu, 13 Oct 2011 19:39:08 -0000

I totally agree on eliminating the POSIX hierarchy as being an official part of the file system. The "/" separator can be demoted to being a human convention that has no meaning to the file system itself, just as ".extension" has been since Unix became dominant.

When hierarchical directories were created they served a real purpose -- they limited how much directory information had to be read at once. Well systems today have a lot more than 64KB of RAM to work with, they can support gigantic flat directories with long file names.

I am less certain about Jeff Darcy's comment. I think the other POSIX element that S3, GFS, HDFS, SWift/etc. have eliminated is partial file update. I think the Facebook Haystack designers hit the nail on the head when they championed a post-POSIX paradigm "where data is written once, read often, never modified, and rarely deleted."

Re: We Need a Storage Revolution

Jay Sharp — Fri, 20 May 2011 06:22:58 -0000

Amen!

Re: We Need a Storage Revolution

Obdurodon — Sat, 30 Apr 2011 13:19:32 -0000

That's exactly my concern too, Steve. I think what the world really needs is a system in which objects are named or found in a more sophisticated way than a POSIX-style hierarchy, but once found have POSIX-like semantics such as efficient random-byte updates in the middle of large objects. In more specific terms, something better than open() and mkdir() but exactly the same read() and write(). In fact, I think you and I and Mich discussed this idea at some length back in 2001 or so. Should've gone out and worked on it. ;)

Re: We Need a Storage Revolution

stevetodd — Mon, 29 Sep 2008 00:29:05 -0000

Stephen,
One concern I have with CAS eliminating block/file is the transactional semantics of changing data. Block and file approaches handle changing data quite well; CAS (and XAM) are geared towards fixed content (unchanging data). But I do agree with the stance that you are taking and the direction that your thoughts are going: concepts like CAS solve problems in ways that block and file cannot.
Steve