SID-00276: TWiki Slow due to many thousands of files in a sub web

Status:	Answered	TWiki version:	4.2.4	Perl version:	v5.8.5
Category:	CategoryPerformance	Server OS:	RHEL4.7	Last update:	17 years ago

Hello - I have a performance problem with a clustered twiki setup (using RH Cluster and gfs). A twiki web has over 26,000 files in a particular directory and every view or edit appears to search the entire data directory structure including a walk of all the files in this directory. Due to file locking on the gfs filesystem every single twiki page view takes up to 10 seconds. I have tested moving the web to a non-clustered twiki environment I have and, even though the directory is still walked in the same way, the performance is acceptable (so gfs/file locking is causing the bottleneck).

An strace of TWIKI_HOME/bin/view produces this (BOOM_BOOST dir has 26,000 files):

10:13:24 open("/var/www/twiki/data/HIT/Specifications/BOOM_BOOST", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 15
10:13:24 fstat(15, {st_mode=S_IFDIR|0755, st_size=2990080, ...}) = 0
10:13:24 fcntl(15, F_SETFD, FD_CLOEXEC) = 0
10:13:24 getdents64(15, /* 58 entries /, 4096) = 4080
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 57 entries /, 4096) = 4080
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:24 getdents64(15, / 56 entries /, 4096) = 4032
10:13:25 getdents64(15, / 57 entries /, 4096) = 4088
10:13:25 getdents64(15, / 56 entries /, 4096) = 4032
10:13:25 getdents64(15, / 56 entries /, 4096) = 4032
10:13:25 getdents64(15, / 56 entries /, 4096) = 4032
10:13:25 getdents64(15, / 56 entries /, 4096) = 4032
10:13:25 getdents64(15, / 56 entries */, 4096) = 4032

When I do the same on the clustered environment, the time to walk the entire BOOM_BOOST dir is about 8 seconds(!).

Does twiki need to reference every file in every web when it displays a single page or is this a local configuration problem? Is this some other type of plugin problem?

Thanks in advance for you help

David

-- DavidBrezynski - 2009-04-21

Discussion and Answer

In my TWiki installations I have a flat web structure and I disable the {EnableHierarchicalWebs} configure setting. This speeds up TWiki with large numbers of pages per web considerably. You can list key webs indented in the sidebar and homepage to give the impression of hierarchy.

Consider a load-balanced setup if you have many users. Optimize the storage server, possibly as simple as mirrored disks if your TWiki data is less then terabytes.

See also TWikiScalability.

-- PeterThoeny - 2009-04-21

Thanks for the information. We don't have a large number of users, and we already have hierarchical webs defined so it might be difficult to back these out. The performance slowdown is definitely a result of every single file in the web being touch each time a view or edit is done. I'd like to understand more about why this happens. Is this expected? Is there any doc that explains why this is necessary?

I'm convinced that the bottle neck is the gfs distributed lock manager (lock_dlm) in the clustered setup as this process begins to take lots of CPU each time a view is run.

Thanks for your time. David

-- DavidBrezynski - 2009-04-22

I am pretty sure that there is no reason to touch all files in a sub-web on a page view or edit. Something that definitely needs to be optimized. TWiki is open source, you and others are invited to investigate and post a patch.

-- PeterThoeny - 2009-04-22

I invite you to get involved with the TWiki community. As a contributor you can actively make a difference and help shape the product. A good way to get engaged is to participate in the biweekly release meetings (monitor the [twiki-dev] mailing list and the Codev.Webhome web changes). To get started on specific areas visit:

• Codev.ReadmeFirst - for core code
• Plugins.ReadmeFirst - for plugins
• Codev.TWikiMarketing - for marketing

Closing this question after more than 30 days of inactivity. Feel free to reopen if needed. Consider engaging one of the TWiki consultants if you need timely help. We invite you to get involved with the community, it is more likely you get community support if you support the open source project!

-- PeterThoeny - 2009-06-02

If you answer a question - or someone answered one of your questions - please remember to edit the page and set the status to answered. The status selector is below the edit box.