SID-01982: Topic with forms take 20 seconds to display in 6.0.1

Status:	Answered	TWiki version:	6.0.1	Perl version:	perl-5.10.1-130.el6_4.x86_64
Category:	CategoryPerformance	Server OS:	CentOS 6.4	Last update:	10 years ago

I have a topic which contains 1 include and 3 search in forms

It used to display quite quickly in twiki4 AND also in twiki6.0.0, on CentOS 6.4.

but since I have installed twiki6.0.1 (and migrated my twiki6.0.0 with BackupRestore), it is now VERY SLOW to display, it takes about 20 seconds...

(I did not change anything in this topic...)

Here are some directives contained in this topic :

%INCLUDE{PcAddNew}%

3 times, I have a search like this (which displays like a table of about 20 lines) :

%SEARCH{ "META:FORM.*PcForm" type="regex" ...

(I'm viewing this on Mac OS 10.7 with Firefox 32, but it was quick with twiki4 and twiki600)

Do you know what could be the problem ?

Not a big deal...

-- Etienne Pallier - 2014-10-17

Discussion and Answer

I've commented the 3 searches in this topic and it displays very fast, so the issue is really with the search.

The 3 searches are quite the same, they all have the same format like this one :

%SEARCH{ "META:FORM.*PcForm"
 type="regex"
 nonoise="on"
 excludetopic="*Template, %TOPIC%, PcList"
 order="formfield(PcPurchaseDate)"
 reverse="on"
 header="| Nom | Type PC | Date Achat | Marque | Modèle | OS | OS Version | (premier utilisateur) | Proc | RAM | HD | @IP | Equipe |"
 format="$percntCALC{$IF( $AND( $NOT($EXACT($formfield(PcType),Serveur)) , $NOT($EXACT($formfield(PcUser),LIBRE)) ), $LISTJOIN($sp, | $formfield(PcName) | $formfield(PcType) | $formfield(PcPurchaseDate) | $formfield(PcBrand) | $formfield(PcModel) | $formfield(OSFamily) |
 $formfield(OSDistrib) | $formfield(PcOwner) | $formfield(PcProc) | $formfield(PcRam) | $formfield(PcHd) | $formfield(PcIP) | $formfield(PcTeamOwner) |), <nop>)}$percnt"
}%

So, what is wrong with twiki 6.0.1 ?

-- Etienne Pallier - 2014-10-17

This is a serious issue that needs to be debugged. How many PcForm topics do you have? What is the page load time with TWiki-6.0.0?

-- Peter Thoeny - 2014-10-17

Some tips on performance tuning:

1. Use CALCULATE instead of CALC where possible. In your case it is possible because you do not reference any table cell, just form field values.

2. Consider using just one SEARCH where you assign form field values to hashes, then create the tree tables using those hashes. Detail at Advanced TWiki Spreadsheet Formulas: Power and Flexibility - Not For Grandma.

On debugging the issue, let's see if it is caused by the SpreadSheetPlugin or SEARCH. Could you install an older version of the plugin? In Plugins.SpreadSheetPlugin, click on manage of a SpreadSheetPlugin.zip, and pick a version from a year ago. Install it on your new TWiki by copying just lib/TWiki/Plugins/SpreadSheetPlugin.pm and lib/TWiki/Plugins/SpreadSheetPlugin/Calc.pm. Let us know what time difference that makes in page load time.

-- Peter Thoeny - 2014-10-17

Peter,

sorry to come back so late to your questions, I was very busy on other projects.

Here are my answers :

I have less than 50 PcForm topics.

I don't remember the load time with twiki600 but it was like 3 to 5 seconds max. Now, it is 20 seconds !!!

-- Etienne Pallier - 2014-11-03

Your proposition :

1. Use CALCULATE instead of CALC where possible. In your case it is possible because you do not reference any table cell, just form field values.

It takes the same time...

-- Etienne Pallier - 2014-11-03

Regarding this proposition of yours :

On debugging the issue, let's see if it is caused by the SpreadSheetPlugin or SEARCH. Could you install an older version of the plugin? In SpreadSheetPlugin, click on manage of a SpreadSheetPlugin.zip, and pick a version from a year ago. Install it on your new TWiki by copying just lib/TWiki/Plugins/SpreadSheetPlugin.pm and lib/TWiki/Plugins/SpreadSheetPlugin/Calc.pm. Let us know what time difference that makes in page load time.

I tested it (took a version of 2013 sept) and it still takes the same time...

-- Etienne Pallier - 2014-11-03

Maybe I should return to twiki 600 ?

-- Etienne Pallier - 2014-11-03

This is now handled at TWikibug:Item7616.

-- Peter Thoeny - 2015-02-26

Please use latest TWiki::Search module, or apply the patch listed at TWikibug:Item7616. Page load time on a complex page with 5 SEARCHes is now cut to 1/3, bringing search speed back to TWiki-5.1 level.

-- Peter Thoeny - 2015-03-04

I downloaded the Search.pm from http://svn.twiki.org/svn/twiki/tags/TWikiRelease06x00x00/core/lib/TWiki/Search.pm

I renamed my current lib/TWiki/Search.pm into Search.pm.ORIG, and put the new version in place

It still takes the same 20 seconds for my form to display...

(I'm using 6.0.1)

-- Etienne Pallier - 2015-03-04

Oops, sorry, I pasted the wrong URL in TWikibug:Item7616, now corrected. Use: http://svn.twiki.org/svn/twiki/branches/TWikiRelease06x00/core/lib/TWiki/Search.pm

-- Peter Thoeny - 2015-03-04

ok, it works now !!! Only 7 seconds instead of 20 seconds before ! Thank you very much Peter

-- Etienne Pallier - 2015-03-06

If you answer a question - or someone answered one of your questions - please remember to edit the page and set the status to answered. The status selector is below the edit box.