SID-00937: Creating large sub-web makes Main.WebHome time out

Status:	Answered	TWiki version:	4.1.2	Perl version:	5.008008 (linux)
Category:	CategoryPerformance	Server OS:	RHEL4 2.6.9-34.elsmp (x86_64-linux)	Last update:	16 years ago

We have a Reports web which has grown too large (98G >200k files). I tried to solve this by creating a Reports.Archives sub-web which hosted the older files. After doing this, the main page ( Main.WebHome) started timing out. Doing an strace on bin/view reveals that the process is doing a listdir operation (getdents64()) on the archive directory, which is taking too long.

1. Is this expected behavior?
2. Is there any way to disable it?
3. Has it been fixed in a later TWiki version?

Thanks!
-- BuckGolemon - 2010-08-10

Discussion and Answer

I do not recommend using sub-webs if you have more than 50k pages in any web. There is a known performance limitation with sub-webs. Try to move the web(s) to a flat list (you can do that on the shell level under twiki/data and twiki/pub) and disable {EnableHierarchicalWebs} in configure.

See also TWikiScalability.

-- PeterThoeny - 2010-08-10

I've done that and performance is back to normal. Thanks.

--Buck

-- BuckGolemon - 2010-08-21

I am glad it worked out. Once we have the DB backend interface we will address this issue.

-- PeterThoeny - 2010-08-21

If you answer a question - or someone answered one of your questions - please remember to edit the page and set the status to answered. The status selector is below the edit box.