Bug: Unresolved variables in the URL cause major performance problems and seemingly unrelated errors.

Symptoms can any one or more of:

a) User hits [edit][save][preview][etc...], the browser makes a request to the server, some data comes back, but nothing is changed. You are still sitting in edit mode, or wherever. If you repeat the command often enough, you will eventually get the expected outcome.

b) In InternetExplorer, you slap [save], browser makes a trip to the server, and sits there waiting forever. If you wait long enough, CPU usage climbs to 99% and the (client) system gets very sluggish.

c) Konqueror may crash on just viewing a page.

d) in the apache log there are errors like:

[Fri Dec 03 16:09:57 2004] [error] [client 199.247.128.35] 
(104)Connection reset by peer:
ap_content_length_filter: apr_bucket_read() failed, 
referer: http://yukongis.ca/bin/view/Sandbox/WhatsWrongIE

e) if you use the apache2 hang fix workaround open(STDERR, ">>/blah/blah") in bin/setlib.cfg there might be errors like:

save: Use of uninitialized value in string eq at ../lib/TWiki/Store.pm line 839.
save: Use of uninitialized value in string at ../lib/TWiki/Store.pm line 659.
save: Use of uninitialized value in substitution (s///) at ../lib/TWiki/Store/RcsWrap.pm line 597.

f) for the browsers which do work (Mozilla/Firefox) you occasionally get in the browser window on save:

/usr/bin/ci -q -l -m"none" -t-none -w"" 
'/usr/local/sites/yukongis.ca/www/htdocs/data/Sandbox/WhatsWrongIE.txt' 2>&1 
ci: missing author for -w option

/usr/bin/ci -q -l -m"none" -t-none -w"guest" 
'/usr/local/sites/yukongis.ca/www/htdocs/data/Sandbox/WhatsWrongIE.txt' 2>&1 
ci: RCS file /usr/local/sites/yukongis.ca/www/htdocs/data/Sandbox/WhatsWrongIE.txt,v is in use

g) If you disable authentication, all browsers can view and edit pages at will.

h) If you set the skin to classic (and perhaps any one other than PatternSkin), the problems go away.

SOLUTION

Define USERLAYOUTURL, USERSTYLEURL and FAVICON in TWikiPreferences to non-empty values (can still be overridden in Web and User prefs). empty.css did not previously exist.

---+++ PatternSkin

   * Using attachments for styles 
      * Set TWIKILAYOUTURL = %PUBURL%/%TWIKIWEB%/PatternSkin/layout.css
      * Set TWIKISTYLEURL = %PUBURL%/%TWIKIWEB%/PatternSkin/style.css
      * Set USERLAYOUTURL = %PUBURL%/%TWIKIWEB%/PatternSkin/empty.css
      * Set USERSTYLEURL = %PUBURL%/%TWIKIWEB%/PatternSkin/empty.css

Special Note: these are all the symptoms I experienced in the last four months. They are not necessarily related to the root problem identified, unresolved variables in the URL. I am no expert. However once I got rid of the unresolved vars, all the symptoms described above went away.

A great big thanks to all those on TWikiIRC who repeatedlty threw their browsers against my wall in an effort to get to the bottom of this.

-- MattWilkie - 08 Dec 2004

My jubilation was premature. It's not fixed. Now I have to (try and) find out what the heck was different between 4 and 5 oclock this afternoon.

me slinks away, crawls into a hole, and pulls it shut

it's still broken, though. i've checked this fix into DEVELOP r3378. sorry to hear it hasn't fixed your problem...

-- WillNorris - 09 Dec 2004

Environment

-- MattWilkie - 08 Dec 2004

Follow up

Just wondering if Apache 2 is the root cause here - from this search it looks like a number of different web applications have the same problem, and all in Apache 2.0. The RCS and browser issues could be consequential on Apache corrupting some data that TWiki or the browser later depends on.

What does 'in the URL' mean? Is this the URL to the CSS stylesheets? If so, let's rename to UnresolvedVarsInStylesheetURL, though we might want to further characterise the issue first.

-- RichardDonkin - 09 Dec 2004

Oh sorry, I didn't post the log entries which finally got me looking at PatternSkin specifically (apache access.log):

"GET /bin/view/TWiki/%USERLAYOUTURL% HTTP/1.0" 400 300
"GET /bin/view/TWiki/%USERSTYLEURL% HTTP/1.0" 400 300
...

However, I have since determined that it is not enough to simply set those vars. All I know for sure is that, on DEVELOP r3355 and 01-Sept-2004, if I a) disable PatternSkin, or b) turn off authentication, all the errors and symptoms described above go away.

-- MattWilkie - 09 Dec 2004

I captured what normally goes to the browser by putting open (STDOUT, ">>/tmp/stdout.log") in setlib.cfg. The resultant page looks like:

Status: 302 Moved
Location: http://yukongis.ca/bin/view/Sandbox/WhatsWrongIE

Content-Type: text/html; charset=ISO-8859-1

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>  WhatsWrongIE &lt; Sandbox &lt; wiki</title>
...etc

why does it have a status of 302 moved?

-- MattWilkie - 11 Dec 2004

I don't know if this is related or not, I think it is, I can't save edits to TWikiPreferences. There is no error message to the browser, just a some traffic on the modem and then nothing.

And trawling through the apache mailing lists and bugzilla (boy, are those ever huge) I saw some references that inidicate large numbers of unresolved variables in the perl cgi's can generate large amounts of stderr which apache shuts down. This is another face of the TWikiOnApache2dot0Hangs bug which has been fixed for a few months now, but I can't help wondering if the sheer number of variables in use by twiki is a problem here.

-- MattWilkie - 14 Dec 2004

Apache 2 isn't the root cause here. The same issue happens with Apache 1.3.33.

-- StevenHajducko - 23 Dec 2004

Someone just pointed me to the Live HTTP Headers extension for Firefox which might help figure this thing out. http://livehttpheaders.mozdev.org/

Initial tests seem to indicate that on my site, authorization is asked for twice on every save, while on twiki.org the auth challenge only happens once. Sandbox.HttpHeaders

-- MattWilkie - 23 Dec 2004

Per the Live HTTP Headers, my site is only asking for authentication once.

-- StevenHajducko - 28 Dec 2004

There are two sorts of 'unresolved variables' that may cause problems:

1. Perl variables such as $foo that are not initialised, flagged by -w switch or use warnings - these should be tracked down and fixed, as they indicate code quality issues and inflate the Apache logs.
2. TWikiVariables, e.g. %FOO% - these may cause interesting issues if in URLs, since %FO may be attempted to be converted to a single byte using URL encoding (e.g. %2F is valid hexadecimal).

-- RichardDonkin - 31 Dec 2004

With IE and HTTP 1.1 turned off, posting works much the same as it does in FF (occasional double posts because of needing to refresh after an 'add comment').

there is a new error in the cgi-stderr.log:

[Sun Jan  2 09:12:32 2005] viewauth: Argument "0\0" isn't numeric in numeric eq (==) at /usr/local/sites/yukongis.ca/www/htdocs/lib/TWiki/Plugins/CommentPlugin/Comment.pm line 297.
[Sun Jan  2 09:12:32 2005] viewauth: Argument "0\0" isn't numeric in numeric eq (==) at /usr/local/sites/yukongis.ca/www/htdocs/lib/TWiki/Plugins/CommentPlugin/Comment.pm line 297.

HttpOneDotOneSupport

-- MattWilkie - 03 Jan 2005

If you want to fix that error, you can modify the Comment.pm and change the "==" to "eq" on line 297.

I wonder why it would show up when switching from 1.1 to 1.0.

-- StevenHajducko - 03 Jan 2005

thanks for fix!

-- MattWilkie - 04 Jan 2005

No problem, although I'm curious to know why turning HTTP 1.1 off would cause the argument to become something non-numeric. Guess it really don't matter though and is getting off-topic.

I still haven't had any issue since making the changes that have the strike-thru above. I'm beginning to think that they do work.

-- StevenHajducko - 06 Jan 2005

Just had this using 02 Sep 2004 release on Windows with Firefox 1.0.1 and Apache 1.3.29 (Cygwin). Seems to happen more on longer pages, but affected every page once it happened.

Worked around this by setting classic skin, since I'm a bit busy and it's only for local note taking.

-- RichardDonkin - 01 Mar 2005

I found Firefox tries to read every page twice, with Accept: header expecting text/css at the second time. This comes from @import url(""). Previous versions silently ignored empty url but latest version treat it as the document url itself.

Setting empty.css to USER*URL seems to solve the problem for me. Why the above empty.css solution is struck out? Are there any problems remaining?

-- KaoruMaeda - 31 Mar 2005

My web host has moved to a new hardware platform and upgraded their OS (some flavour of redhat I believe) and most if not all of my problems have disappeared. I'll try and refactor this collection of topics into some kind of coherency after I get caught up with all recent happenings on twiki.org and irc.

-- MattWilkie - 31 Mar 2005

Kaoru, I struck out the empty.css solution as it didn't solve the problem which led me to find the unresolved URLS in the first place: WhatsWrongIE. I now believe that the unresolved urls just exacerbated a pre-existing problem, which thankfully, now seems to have disappeared. My web host changed their server. In the process the linux kernel was also upgraded to 2.6.10-1.770_FC3smp and perl to 5.8.5 (and likely some other changes too).

So for clarity, setting the prefs fixes one problems listed above.

I've decided to leave most of the mess this page is in intact as I don't know how to tease the component unresolved bugs apart.

-- MattWilkie - 20 Apr 2005

from UsingTopicToDefineCSS: "But problems arise when one of the variables is not defined. Then we get @import url(""). Some browsers (at least Mozilla and Firefox) then read the same topic twice (presumably try to read the page url as css file - resulting in performance loss. And when se set the view to authenticated users only, they need to enter a password twice. Very annoying. I've tried to use css file linking with the syntax <link rel="stylesheet"... but that results in the same behaviour. If you use @import url("none") you will get a log message that the file does not exist. And I suppose this also leads to a small overhead. I can see 2 solutions to our problem:

1. the @import line needs to be a conditional, and will not be shown if USERLAYOUTURL is empty
   • This could be done when javascript is used to write the style imports
2. an empty USERLAYOUTURL should fall back to a default (existing) file, for instance PatternSkin's empty.css.
   • Then we need to pass this file as parameter to RELATIVETOPICPATH.

-- ArthurClemens - 23 Apr 2005"

-- MattWilkie - 24 Apr 2005

See also the solution in UsingTopicToDefineCSS.

-- ArthurClemens - 24 Apr 2005