Question

Hello Twiki Users support forum. I am encountering a problem with a Twiki upgrade, and I have reached the point where it seems appropriate to ask for help. Let me first say that I have read all of the documentation that I can find on this issue; not that I haven't most likely overlooked something, but that I have tried to be a concientious member of the Open Source community, and that I appreciate any help I can get on getting over this stumbling block. I intend to be verbose in the interest of providing all of the important details to anyone who wants to give me a hand.
So here are the hairy details: I am attempting to upgrade from TWiki 02 Sept. 2004 (which I believe is Cairo) to Dakar. I intially tried the UpdateTWiki script, but that appeared to work incorrectly for me. Therefore I resigned to a manual migration. I unpack and install/configure Dakar(including touching LocalSite.cfg and LocalLib.cfg); at this point everything is working fine, as I can freely edit topics (well, the left navigation bar appends itself to the bottom of the page. Not sure why that is, but it's most likely grist for a seperate ticket).
The bin/configure page reports no errors. However once I tar over the data and pub directories and run the unlock script provided with the documentation, it appears that my files have not been unlocked, as I cannot open the for editing and the warn logfile in /data reports:
RCS: stream open /usr/share/twiki4.0.2/pub/Deploy/WebHome/_filetypes.txt failed: No such file or directory at /usr/share/twiki4.0.2/lib/TWiki/Store/RcsFile.pm line 832. I attempted the alternative method suggested of symlinking to the target directories, and achieved the same result.
Reading the available documentation here under Support/AnsweredQuestions I found this link: http://koala.ilog.fr/twikiirc/bin/irclogger_log/twiki?date=2006-02-11,Sat&sel=55#l51 where it is suggested that perhaps I have not declared $TWIKIROOT$/bin to be a cgi-bin to Apache. I'm not sure if this applies to my case however, as Dakar works before the importation of the old directories; this indicates to me that cgi is running correctly. I have the line ScriptAlias /twiki4.0.2/bin/ "/usr/share/twiki4.0.2/bin/" in my httpd.conf file.
RCS doesn't recognize the -r flag in CentOS4.0. I found suggestions to run RCS without the flag within the TWiki documentation relating to upgrades. Sample output of the rcs -u -M '{}' /; command follows: RCS file: pub/AM/PhonesPrioritized/CopyofPhonesprioritized2.xls,v Revision 1.1 is already locked by apache. 1.1 unlocked done
I additionally tried running this script with Apache turned down (in case its lock was breaking things) to no avail. I have attached the output of my configure page here; I am utterly stumped and appreciate any help I can get.

Environment

-- JosephHammerman - 20 Apr 2006

Answer

You can drop the -r flag, the unlock command works fine without it.

As long as the topic has been unlocked (you see the message "1.1 unlocked" or similar) then you have done the right thing. I don't think that error message is anything to so with the RCS locks. Are your permissions correct (the apache user has to be able to read/write all files). Are your data and pub directories on a different disc? NFS mounted? Using a different format e.g. NTFS? What is CentOS4.0? Is it a Linux? If so what kernel version? Does the _filetypes.txt file exist? is there a _filetypes.txt,v file with it?

-- CrawfordCurrie - 20 Apr 2006

If you answer a question - or someone answered one of your questions - please remember to edit the page and set the status to answered. The status selector is below the edit box.

1. Yes, the apache user has read/write access to all files, and in my httpd.conf it is declared that httpd shall run as Apache user and Apache group. 2. The data and pub directories are on the same disk and partition as the new $TWIKIROOT$, on ext3fs filesystem type. 3. CentOS4 is a rebuild, from the released source, of RHES 4.0. It's like community edition RedHat Enterprise system. 4. My Kernel version is 2.6.9-5.0.3.ELsmp. 5. No _filetypes does not exist. There is no versioned filetypes either. Is it worth mentioning that there is no _filetypes file in the source directory either?

Thanks for your help!

-- JosephHammerman - 20 Apr 2006

Your files have been unlocked, AFAICT. For some reason you are trying to access this _filetypes.txt file that you have said does not exist. I have no idea why this file is being referenced. it is not used by TWiki, AFAIK. At least I don't recall ever having seen any reference to it.

Please try isolating the problem; pick one file that you converted over; try and edit and save it; and tell us all the error messages from you apache error log that ensue.

-- CrawfordCurrie - 20 Apr 2006

Ok, I tried to edit I.T. KickStart guide. Here is the error message from /var/log/httpd/error_log:
[Thu Apr 20 12:47:28 2006] [error] [client 172.16.130.217] RCS: stream open /usr/share/twiki4.0.2/pub/IT/WebHome/_filetypes.txt failed: No such file or directory at /usr/share/twiki4.0.2/lib/TWiki/Store/RcsFile.pm line 832., referer: http://ops.corp/twiki4.0.2/bin/view/IT/KickstartGuide And here is the error from $TWIKIROOT$/data/warn200604.txt
| 20 Apr 2006 - 12:47 | RCS: stream open /usr/share/twiki4.0.2/pub/IT/WebHome/_filetypes.txt failed: No such file or directory at /usr/share/twiki4.0.2/lib/TWiki/Store/RcsFile.pm line 832. If you run bin/edit from the command line as the superuser, you get somewhat more descriptive data:
[Thu Apr 20 12:53:11 2006] edit: Use of uninitialized value in pattern match (m//) at /usr/share/twiki4.0.2/lib/TWiki.pm line 1109. ****************************** RCS: stream open /usr/share/twiki4.0.2/pub/Main/WebHome/_filetypes.txt failed: No such file or directory at /usr/share/twiki4.0.2/lib/TWiki/Store/RcsFile.pm line 832.

****************************** Content-type: text/plain

RCS: stream open /usr/share/twiki4.0.2/pub/Main/WebHome/_filetypes.txt failed: No such file or directory at /usr/share/twiki4.0.2/lib/TWiki/Store/RcsFile.pm line 832. TWiki detected an internal error - please check your TWiki logs and webserver logs for more information. I ran locate _filetype; This file exists in the original installation of twiki which I am upgrading from; here is some sample content:
shared sniff
shtml html
sniff sniff
tar zip
taz zip
text txt
tgz zip
tif bmp
tiff bmp
ttf ttf
txt txt
tz zip
vbs mov
wav wav
wri wri
xbm bmp
xla xls
xlb xls
xlc xls
xld xls
xlk xls
xll xls
xlm xls
xls xls
xlt xls
xlw xls
xml xml
xsl xsl
z zip
zip zip
Interestingly, $TWIKIROOT$/pub/IT/WebHome does not exist in either the source or target directory. Why is Twiki looking for this non-existent directory? If I manually add the file to the webhome of the topic I attempting to edit (when it is Main, which exists), then the edit page appears dutifully and everything appears to be functioning normally. Shall I just recursively copy this file into all the WebHomes I can find? Shall I create the missing directories? Final note (and again, thank you for your assistance): if you take a look at the link I posted in my original support request here:
http://koala.ilog.fr/twikiirc/bin/irclogger_log/twiki?date=2006-02-11,Sat&sel=55#l51
one responsder in the IRC room somehwat cryptically suggests that he had the same problem:

Rhasta check your error_log i was getting that for missing _filetypes.txt files after upgrading.

-- JosephHammerman - 20 Apr 2006

./pub/TWiki/TWikiDocGraphics/_filetypes.txt -- not that i know exactly what it's for (yet)

-- WillNorris - 20 Apr 2006

it's used for ObjectMethod mapToIconFileName( $fileName, $default ) -> $fileName, FWIW

-- WillNorris - 20 Apr 2006

Follow up: testing reveals that if I must manually add the file _filetypes.txt to newly created WebHomes as well.
$TWIKIROOT$/pub/<webname>/<WebHome>/,
and I must create the WebHome directory. Any ideas why TWiki would be looking in a directory that doesn't exist for a file that doesn't exist?

-- JosephHammerman - 20 Apr 2006

Follow up: migration to RHES4.3 and attempting to upgrade on the new server results in the same error message.

-- JosephHammerman - 21 Apr 2006

From reading the code, _filetypes.txt has to exist as an attachment to whatever topic you specify in * Set ICONTOPIC. So find where you set ICONTOPIC, and check that _filetypes.txt exists in the corresponding pub directory, and is readable.

The chances are good that you have imported an overriding setting of ICONTOPIC. My recommendation would be to remove all local settings of ICONTOPIC and only leave the default setting. You do not need to add it to WebHomes if there is a default setting in TWiki.TWikiPreferences (or Main.TWikiPreferences)

-- CrawfordCurrie - 21 Apr 2006

Ok, I copied out the instance of TWiki.pm in the target directory (Dakar), copyied over of the pub and data directories as described in the documentation, and then copied back in TWiki.pm to it's original location. Everything seems kosher now as relates to the problem which caused me to create this ticket. So thank you very much for your help. I still have one problem, which you can hopefully assist me with: It seems that Pattern skin is not working correctly. I have attached a screenshot of the Main page so that you can see that the skin is not installed, and that left menu bar appends itself to the bottom of the screen.
When I direct my browser to webroot/bin/view/TWiki/PatternSkin?skin=pattern, I recieve a page identical to the attached in terms of layout and style. Clicking the "Click for full screen image" link produces a 403 forbidden error
Forbidden
You don't have permission to access /twiki4.0.2/pub/TWiki/PatternSkin/patternskin_screenshot_full.gif on this server.
The server log reports: "client denied by server configuration: /usr/share/twiki4.0.2/twiki4.0.2, referer: http://twiki/bin/view".
From reading the documentation that I can find on this issue, this is usually due to a error in server permissions. But I believe I have everything configured correctly.
ls -las on the file in question reports:
$sudo ls -las /usr/share/twiki4.0.2/pub/TWiki/PatternSkin/patternskin_screenshot_full.gif
68 -rwxrwxr-x 1 apache apache 63480 Aug 17 2004 /usr/share/twiki4.0.2/pub/TWiki/PatternSkin/patternskin_screenshot_full.gif
Finally, here is my virtual host configuration for the TWiki webserver:
<VirtualHost 172.16.3.7>
ServerName twiki4.0.2.mobitv.corp
DocumentRoot /usr/share/twiki4.0.2/
ScriptAlias /twiki4.0.2/bin/ "/usr/share/twiki4.0.2/bin/"
<Directory "/usr/share/twiki4.0.2/bin/">
Options +ExecCGI
SetHandler cgi-script
AllowOverride All
Allow from all
Require valid-user
</Directory>
<Directory "/usr/share/twiki4.0.2/pub/">
Options FollowSymLinks +Includes
AllowOverride None
Allow from all
</Directory>
<Directory "/usr/share/twiki4.0.2/data/">
deny from all
</Directory>
<Directory "/usr/share/twiki4.0.2/templates/">
deny from all
</Directory>
</VirtualHost>
Thanks again!

-- JosephHammerman - 24 Apr 2006

Two possibilities:

1. You have a .htaccess file in pub somewhere
2. Your permissions are too generous. Some virtual server configs require that files are not writable by anyone but the apache user.

-- CrawfordCurrie - 24 Apr 2006

Hrm. I commented out the .htaccess file in pub/_work_areas/.htaccess. I HUPed the httpd server and the behaviour did not change. I ran the command sudo chmod -R 755 ./* from the webroot, and I then ran the command sudo chown -R apache.apache from the same location, and HUPed the server once again. Again the behaviour has not changed. Any other ideas I could try out? Have you noticed the weird path in the server error log? That directory does not exist. Perhaps I should post the httpd.conf file as well?

-- JosephHammerman - 24 Apr 2006

Sorry, Irealized that my above post was somewhat cryptic. Specifcally, the web root is /usr/share/twiki4.0.2, and the server error log appears to be looking for files in /usr/share/twiki4.0.2/twiki4.0.2. Also, the .htaccess file I commented out was the only file underneath the pub directory.

-- JosephHammerman - 24 Apr 2006

Ok, the web root should have been /usr/share not /usr/share/twiki4.0.2. The Left hand sidebar is still appended to the obttom of the page.

-- JosephHammerman - 24 Apr 2006