[lug] Too many files error

[David L. Anselmi]

Ken Kinder wrote:
> I'm working on an application that creates lots of files.

Why?  Can you combine your data so it can go into fewer files?  Berkeley DB?

> To reduce the number of files in one dir, I have broken the files up
> into subdirs. The files are basically md5 hex digests, and I'm taking
> the first five chars to determine a directory:
> 
> xxxxx/(file)
> 
> However, I'm running into "Too many links" as an error upon creating
> the directories. This is happening right after the 32,0001st
> directory, which seems like an awfully even number. I thought ext3
> didn't have this problem.

http://lkml.org/lkml/2004/7/8/157

So it looks like you have to use only 3 digits as the directory name. 
But you can do more than one layer in the hierarchy.

You didn't say what kernel you're using so the above may not apply for 2.6.

You could also switch to Reiser or others.

Dave