Installing TWiki on Dreamhost

---

Installing TWiki 4.1.2 on dreamhost

Note (1 May 07): There are excellent instructions directly on DreamHost support wiki at: http://wiki.dreamhost.com/TWiki#Installing_Twiki_on_Dreamhost_on_mydomain.com.2Ftwiki

I finally had to do a clean install, 4.1 is much much easier than earlier versions, and now essentially functions as per the documentation. The configure script now does most of the heavy lifting, so almost all the instructions listed after 'old install notes pre 4.0' aren't needed anymore. I install everything from a dreamhost shell prompt, if you're not comfortable with this maybe twiki isn't for you...

1. download the zip: 'wget http://twiki.org/p/pub/Codev/Release/TWiki-4.1.2.zip'
2. move the zip into your web folder (~/yourdomain.com), unzip into a folder, say 'wiki'
3. by default the bin folder is disabled, so you need to explicity enable script to work. The fastest way to do this is with an .htaccess file. There is one provided with the zip file, called .htaccess.txt. All you need to do is delete the .txt extension.
4. you can now access the configure script from a browser: 'http://www.yourdomain.com/wiki/bin/configure'
5. in the configure script, open 'general path settings'. You'll see several paths in the format '/home/.something/username/yourdomain.com/wiki/{folder}'. delete the '.something', so that it reads '/home/username/yourdomain.com/wiki/{folder}'.
6. the other thing that needs changing is the temp file directory. This needs to point to a folder in your home directory, so make it something like '/home/username/tmp/twiki', and save changes
7. once back in the main configure window, you can scroll down to the plugins section and enable the wysiwyg plugin, save changes once more.
8. i use the twiki-based authentication, so enable that in 'security setup->authentication', and set login manager to template login. You'll then need to point it to an appropriate .htpasswd file to save the user data, this is further down under htpasswd-filename. from memory this defaults to wiki/data/.htpasswd.
9. back in your shell prompt, make the corresponding temp directory: 'mkdir ~/tmp/twiki'
10. that should be it, you should be able to view your twiki by going to http://www.yourdomain.com/wiki/bin/view
11. follow the other steps outlined in the regular documentation
12. Be sure to install the http://twiki.org/cgi-bin/view/Plugins/BlackListPlugin, once you copy the files to their appropriate locations, enable it via the configure script.
13. once you're happy with your twiki installation, it might be a wise idea to disable the configure script, which lives in your wiki/bin folder: 'chmod 444 bin/configure'
14. for added security, you can write-protect your system configuration. the configure script stores your setup in lib/LocalSite.cfg, so set this to be read-only too: 'chmod 444 LocalSite.cfg'.
15. If you need to re-run the configure script again, reset the permissions for configure and LocalSite.cfg ('chmod 755 configure', and 'chmod 644 LocalSite.cfg'). I'm slightly paranoid that way, but my twiki has been hacked a few times now, I'd rather not take any chances...

Old Install notes pre 4.0

Here are some setup notes for TWiki on Dreamhost, who are good for TWiki hosting (Telnet/SSH access, pre-installed RCS, SecureSetup with suexec, extra TWikiSecurity using http://www.modsecurity.org/ mod_security) - see comments at TWikiOnWebHostingSites.

You can try out the resulting site at http://donkin.org, which is normally quite stable but a bit out of date as I have other sites at Dreamhost for experimentation. Check http://donkin.org/bin/testenv for the exact setup I use.

These notes work fine for TWikiRelease02Sep2004 (which has a critical security patch, see TWikiSecurityAlerts for details) as well as TWikiRelease01Feb2003 and Dec 2001.

Dreamhost now have Perl locales working 'out of the box', so testenv should not report any errors on Internationalisation (I18N). This makes them a good choice for anyone who wants to try the TWiki internationalisation features.

If you decide to use Dreamhost, please click on this rewards link or the one on my site (or just email me via RichardDonkin with your userid once signed up) - it helps to pay for hosting four TWiki development sites at present, and is currently the only way that I get some modest funding for my development work on TWiki, which includes InternationalisationEnhancements and TWikiSecurity. -- RichardDonkin

Other guides

There's another, similar, guide here that could also be helpful: http://g-b.dk/Main/InstallTwikiOnDreamhost -- TorbenGB

#Upgrading

Upgrading to v4.04 on dreamhost

I recently upgraded to v4.04 on dreamhost, here's the notes I made while doing so. Seemed to go smoother than the previous version, main problem was permissions not being set correctly from the expanded tar file. The finished site is here: http://www.tokeru.com/twiki Any questions email me at matt.estela@gmailPLEASENOSPAM.com -- MattEstela - 24 Sep 2006

The entire install works as explained in the setup guide, which is nice. So to summarise:

• copy the example .htaccess files correctly, chmod 644 'em, edit them to see if any need specific folders checked (only 2 do from memory)
• configure bin/.htaccess so that you have a valid .htpassword. again, watch the 644!
• edit bin/LocalLib.cfg so that $twikiLibPath is correct. And 644 it.
• should be able to use the online twiki/bin/configure tool now, can set rest of the paths in there.
• Check the general path settings even if it doesn't have errors, they can all point to the wrong things, but won't be flagged.

Enyhoo, should work now if you go /bin/view.

If the twiki works but css/images are missing, might be permissions in the pub folder. there's many files and folders, folders should be 755, files should be 644. Easiest way to do this is:

find . -type d -exec chmod 755 {} \;
find . -type f -exec chmod 644 {} \;

Finally if you're upgrading a wiki, a lot of the skin stuff seems to be broken. I had to revert to the regular pattern skin, then found a lot of the layout was broken. This then involved copying any files matching

*attern*

-- MattEstela - 24 Sep 2006

Background

Dreamhost have a somewhat complicated but very flexible structure of accounts, userids, domains and services - by default you just have one account, one userid (same name as account), one domain, and web (HTTP) service enabled.

• The account gives you a certain amount of bandwidth and disk space, with services such as HTTP.
• Each account can have several userids - by default you just have one userid.
• Each account can be used to host several domains (e.g. example.com) - they appear under your main userid's home directory in their own subdirectories, e.g. example.com

Setup details

Pathnames of the form /home/.aurey/rdonkin should be avoided in favour of /home/rdonkin - the former ties the path to a particular file server, which means that your site may break when Dreamhost occasionally move accounts between file servers to balance the load (mentioned in a recent newsletter). The latter format is used in the notes above and will always work.

Enabling SSH

You first need to enable Telnet/SSH access from the Dreamhost https://panel.dreamhost.com web control panel, specifically by changing your userid from FTP access to Shell access (i.e. Telnet/SSH) on the https://panel.dreamhost.com/?tab=users&subtab=users Users > Users page. If you want to also use FTP access, add another user for FTP only, but note that SCP and SFTP (the SSH file transfer commands) are much more secure, and more convenient once you are used to them - just get hold of a suitable SSH client, search the https://kbase.newdream.net knowledge base for links.

Note that your telnet/SSH password is likely to be different to your main account password, but you can normally change them to be the same.

: For more about SSH configuration, see SshConfig.

Installing TWiki

The key steps that I used are listed here - see TWikiInstallationGuide for more info:

• Unzipped the TWiki files under your domain directory (e.g. example.com or donkin.org)
• Got the real pathname by typing /bin/pwd - used for $homeDir in TWiki.cfg.
• Edited TWiki.cfg to fit, including setting the spamproofing option since this is a public site. Your domain and username will of course be different - note that the simplest way to refer to your home directory, /home/yourname, is actually a symbolic link to something like /home/.aurey/yourname. You should use the /home/yourname format, which is somewhat neater looking and avoids problems if Dreamhost move your home directory:

#                   /cgi-bin/view/Main/WebHome : URL for 'TWiki home', used by TWiki logo in upper left corner :
$wikiHomeUrl      = "http://donkin.org/";
#                   URL for TWiki host :    (e.g. "http://myhost.com:123")
$defaultUrlHost   = "http://donkin.org";

# Home directory for domain
$homeDir = "/home/rdonkin/donkin.org";

#                   /cgi-bin : cgi-bin URL path for TWiki:
$scriptUrlPath    = "/bin";
#                   /p/pub : Public data URL path for TWiki (root of attachments) :
$pubUrlPath       = "/pub";
#                   Public data directory (file path not URL), must match $pubUrlPath :
$pubDir           = "$homeDir/pub";
#                   Template directory :
$templateDir      = "$homeDir/templates";
#                   Data (topic files) root directory (file path not URL):
$dataDir          = "$homeDir/data";

• Did chmod go-w for all files, i.e. find . -print | xargs chmod go-w
  • This is very important for the CGI scripts and 'bin' directory, because Dreamhost uses suexec for SecureSetup, and this is very sensitive to the details of permissions, for security reasons. A bit later, I made the scripts group-writeable and all script execution failed with a 'premature end of script headers' in the error log and 'Internal Server Error' on the web page. Very confusing until I saw the support page on Internal Server Errors.
  • To avoid permissions problems: ()
    • Type ls -l from within your CGI bin directory, and check that the owner (you) has rwx or r-x on every script, and that group and other have r-x or r--.
    • If any permissions are wrong, type chmod u+rwx,g-w * within this directory.
    • (Optional) Type chmod go-rx * within this directory to ensure nobody else can read or execute your scripts.

• Removed old .htpasswd file from data directory (security hole really)
• Edited TWikiPreferences by hand to set WIKIWEBMASTER and set blank SMTPMAILHOST (since Net::SMTP is installed but don't want to use it).
• • You could do this step through TWiki before removing the .htpasswd file of course.
• Re-locked the distributed TWiki files to my userid (rdonkin), since that userid is the one used to run the TWiki scripts ( NOTE: If you are using the most recent testenv, either from SVNget:bin/testenv or TWikiRelease01Sep2004 or later, you can just use testenv to do the re-locking, which is much easier):

   $ *cd data*
   $ *perl -pi~~~ -e 'NR <= 10 && s/nobody:/rdonkin:/ ' */*,v*

• • You can safely remove the .txt,v~~~ files - you might want to do echo *.txt,v~~~ before doing rm -f *.txt,v~~~ (use command line editing to get latter from former, and of course be sure not to enter spaces). Doing a full backup of your *.txt files using tar is a good idea (tar preserves permissions nicely). -- RichardDonkin - 14 Dec 2004
• Some .htaccess hassles, the main thing is to include a top-level .htaccess under your domain directory that looks like:

# The first path here must be a URL path, not a file pathname
Redirect /index.html http://donkin.org/bin/view

Options -Indexes

You should also have .htaccess files in all directories other than 'bin', i.e. data, lib and templates, like this one:

deny from all

My .htaccess in bin starts off like this - the SetHandler line is what enables you to avoid renaming the TWiki scripts to have .pl endings:

# The first path here must be a URL path, not a file pathname
Redirect /index.html http://donkin.org/bin/view

# The path here must be a file pathname, not a URL - first part should match
# the $dataDir setting in TWiki.cfg
AuthUserFile /home/rdonkin/donkin.org/data/.htpasswd
AuthName ByPassword
AuthType Basic

# No changes necessary to this line unless you are using mod_perl
SetHandler cgi-script

# The path here must be a URL path, not a file pathname - first part should
# match the $scriptUrlPath in TWiki.cfg
ErrorDocument 401 /bin/oops/TWiki/TWikiRegistration?template=oopsauth

...
...

Other notes:

• I did a symlink to vim, since a much older vi is first in the path, and vim is much nicer.
• Testenv has a slight bug, in that it mistakenly warns that the templates directory should not be writeable by 'rdonkin' - this is not really a security hole, on Dreamhost at least, because of the SecureSetup using Apache's suexec, meaning that the TWiki scripts are run as 'rdonkin'.

I fixed a couple of warnings in TWikiAlphaRelease of 4 May 2002 that didn't cause any problems other than Apache error log messages - now in CVS, see this http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/twiki/twiki/lib/TWiki.pm.diff?r1=1.138&r2=1.139 TWiki CVS change for details, but note that these are optional unless you have a high-volume site.

Improved security with mod_security (default)

NEW SECURITY FEATURE: Dreamhost now support mod_security (called 'Extra Web Security in their control panel, and now the default). This is an option when configuring your domains that filters many common types of attack including the recent TWikiSecurity alert re the search hole. Seems to work fine, so enable it if you signed up before about Mar 2005, or just leave it enabled if you started using Dreamhost more recently. There's some more discussion at SecureSetup.

This sort of new feature is a big reason I like Dreamhost - they are always coming up with something new (e.g. http://jabber.org Jabber for standards-based http://www.dreamhost.com/rewards.cgi?rdonkin/panel/features.html#jabber instant messaging, WebDAV for http://www.dreamhost.com/rewards.cgi?rdonkin/panel/features.html#webdav easy updates of non-TWiki files, and spam filtering using Razor) as well has having nice standard features like suexec for SecureSetup and IMAP for better email access.

-- RichardDonkin - 14 Dec 2004, 27 Mar 2005

Just a quick note to report that I've had mod_security turned on for my TWiki installation on Dreamhost for several weeks now and have noticed no problems.

-- LynnwoodBrown - 24 Jan 2005, TorbenGB - 27 Jan 2005

Secure site (SSL) setup ()

For people wanting SSL access to their website - it's not obvious from the Dreamhost sales site, which makes it look like a high-end package is needed, but you can buy the low-end package (Crazy Domain Insane, $7.95 per month upwards) and still run an SSL site. The trick is to add a 'Static IP' package, costing $5 per month (from memory), which then enables you to add an SSL ( https) service to your existing domain name.

I use this for https://donkin.org/bin/view - no TWiki changes needed! I only used a free certificate from http://cacert.org, so you'll get a warning in your browser.

You may also want to update the .htaccess file in your domain subdirectory (e.g. ~/donkin.org/ in my case) to include a redirect using https - otherwise, https://yourdomain.com redirects to http://yourdomain/bin/view. Something like this should work, but note that it affects http:// URLs as well:

Redirect /index.html https://yourdomain.com/bin/view

-- RichardDonkin - 27 Mar 2005

Known issues

If you are installing Perl 5.8 from source on Dreamhost, have a look at BuildingPerlCwdIssue.

-- RichardDonkin - 30 Jan 2004

Shared Setup

I recently posted a support topic about MultipleInstallationsUsingSingleTWikiGuts that, so far, has not generated any nibbles of interest. Since this question specifically relates to Dreamhost and its ability to host multiple domains with a simple directory structure, I thought I'd cross-post a note here to see if anyone else on Dreamhost is interested in exploring this further. The basic idea is to have one set of TWiki scripts serving several distinct TWiki installations under separate domains.

-- LynnwoodBrown - 14 Oct 2003

I don't see a problem with doing this, on Dreamhost or other setups - it's just a matter of setting TWiki.cfg appropriately, though you may end up with symbolic links everywhere since TWiki is not really designed to make this easy.

-- RichardDonkin - 14 Oct 2003

Comments

Richard, many thanks for providing an excellent set of notes. They are very easy to follow and very accurate. Particularly good of you to note the Dreamhost suexec situation re chmod changes required.

-- RossC - 19 Jul 2002

Just upgraded http://donkin.org to latest TWikiAlphaRelease, and the permissions thing bit me again - Dreamhost's setup gives a fairly useless message in the browser and the Apache error logs. So the first thing to do if you get problems is:

• Type ls -l from within your CGI bin directory, and check that the owner (you) has rwx or r-x on every script. If not, type chmod u+rwx * within this directory.
• (Optional) Type chmod go-rwx * within this directory to ensure nobody else can read your scripts.

-- RichardDonkin - 19 Nov 2002

Installing TWiki20021230beta, I created first the .htaccess through the Dreamhost panel and modify it exactly as Richard wrote. But the annoying thing was when I modified it after, the Dreamhost panel rewrote it. It's probably better to tune the .htaccess manually with http://httpd.apache.org/docs/howto/htaccess.html to help.

-- BenoitFauvel - 24 Jan 2003

Dreamhost have fixed the Perl 5.6 locale setup, at least on my server ( fud.dreamhost.com), so you can now run TWiki with I18N features fully working in the default $localeRegexes = 1 mode (i.e. WebIndex sorting and case-insensitive searching will work). See http://donkin.org/bin/view/Test/WebIndex my WebIndex page for an example of locale-aware sorting.

To check if your host has been fixed, run testenv and check that it can set the locale correctly - if not, log a call with Dreamhost and comment here. I have some other test programs that demonstrated the problem, so let me know if you have problems, but I suspect this is fixed on all Dreamhost servers by now.

-- RichardDonkin - 25 Sep 2003

Very helpful comments. I was able to get a faint heartbeat by following these instructions. (See http://www.webfoot.com/twiki.)

I also needed to mv data/TWikiRegistrationPub.txt to $data/TWikiRegistration.txt. See the comments in those files.

Hmmpf. I also am having trouble with the authentication. It looks to me like .htaccess and TWiki.cfg are both pointing to the same .htpasswd file, but something still isn't working.

BTW, the site in question is http://www.webfoot.com/blog/

-- DuckySherwood - 29 Mar 2004

Okay, I tried again, and I still had trouble with registration. Somebody didn't seem to be seeing the .htpasswd properly. (Also, there's no place to enter a password on the default form!) I did manage to get a usable account for me by using some brute force (vi), but it took some doing.

-- DuckySherwood - 20 Apr 2004

Dreamhost are doing another sale of web hosting - 50% off one of their higher-end packages including SSL, click here for details. Not sure how long this will go on for, but it's good if you want a more secure TWiki site using SSL (see Google:SSL+TWiki).

-- RichardDonkin - 25 May 2004

Anyone any success/experience of using SpeedyCGI or anything that would speed up an installation on Dreamhost? Thanks.

-- MartinCleaver - 14 Feb 2005

I tried using SpeedyCGI on Dreamhost and it half-worked - seemed to run into intermittent problems. However, that was some time ago so the problem may well have been fixed - I didn't have time to track it down. Dreamhost and other hosting providers may have policies about leaving processes running for a long time, but as long as SpeedyCGI times out after a while it should be OK. Dreamhost doesn't support ModPerl - more expensive 'virtual dedicated server' type hosting would be needed to enable this.

-- RichardDonkin - 27 Mar 2005

thanks very much for these notes!! i've just installed a site at http://notes.prevolution.org using these guidelines, and had absolutely no trouble at all. however, i've just tried to create a new web and i keep on getting a 503 error:

Service Temporarily Unavailable
 The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

so first i figured maybe i'd got the file permissions wrong, but they're all as indicated above and changing the file perms didn't help at all. does anyone have any suggestions?

-- TWikiGuest - 11 May 2005

I believe that I've seen that error message - but the web was still created. You might check to see if your new web exist.

-- LynnwoodBrown - 11 May 2005

Richard, how did you get SpeedyCGI on Dreamhost? They do not have it installed. Did you compile it yourself there?

-- ThomasWeigert - 12 May 2005

I just compiled SpeedyCGI myself - it didn't work well back then, but Debian at least on my server has been upgraded, and no doubt SpeedyCGI has been as well.

-- RichardDonkin - 19 May 2005

Richard, I still don't understand... are you saying that you compiled it on the dreamhost machine? I tried to compile SpeedyCGI on dreamhost but could get nowhere...

-- ThomasWeigert - 13 Jun 2005

With Dreamhost it is easy to give a directory WebDAV access. But any idea how to configure WebDAVPlugin for Dreamhost?

-- ArthurClemens - 23 Jul 2005

I looked into using WebDAVPlugin a while back but it requires rebooting the server and perhaps other things that we can't do for Dreamhost sites.

-- LynnwoodBrown - 25 Jul 2005

Dreamhost wouldn't be able to support WebDAVPlugin, because that relies on an Apache module ( mod_dav). Installing such modules requires administrator access to Apache, which shared web hosting sites such as Dreamhost don't provide.

Dedicated servers are the way to go if you can afford them. Otherwise, virtual dedicated servers (VDSs, see http://www.mythic-beasts.com/vds.html one overview and Google:virtual+dedicated+server) provide the same capabilities (your own virtual Linux box that you can reboot and administer) - they are not much more expensive than some shared hosting plans, though I've not tried them.

A VDS would also enable you to run ModPerl - if you can find sites that do mod_perl hosting they may turn out to be VDSs that also enable installation of new Apache modules. The skills required for VDSs are Linux and Apache admin, which are somewhat more than the typical AdminSkillsAssumptions.

-- RichardDonkin - 31 Jul 2005

AutomaticAttachments and Dreamhost WebDAV offers a tantalising promise: to be able to use the attachment area as a drag target from DAV client such as Windows explorer. However, files uploaded are not owned by the web user but by some DH webdav account.

Any thoughts on workarounds?

-- MartinCleaver - 06 Dec 2005

2nd DH thought for today: Dreamhost also offer FastCGI in place of ModPerl

-- MartinCleaver - 06 Dec 2005

Another thought: be careful about overloading their servers. "Misbehave" and they will move to you one of the DreamhostLimboServers

-- MartinCleaver - 07 Dec 2005

If you want a domain other than .com, .net or .org, you'll need to:

1. Register this domain (e.g. yourdomain.fr) with a national domain name registrar
2. Host the DNS service for this domain with Dreamhost (point the registrar's control panel to use Dreamhost's DNS servers, i.e. ns1.dreamhost.com etc - see Dreamhost Wiki for details, but this is very easy)
3. Host the Web/HTTP service for this domain with Dreamhost - the way to do this is to Add the domain under Dreamhost control panel, then specify yourdomain.fr as the domain (even though registered elsewhere), with a suitable FTP User (can be same as another domain already hosted at Dreamhost) and subdirectory (I suggest yourdomain.fr)

I'm posting this because I've just done it, and the documentation was quite confusing despite the fact that the actual process is very simple.

-- RichardDonkin - 17 Jul 2006

There is an apparent bug in the TWiki 4.1.2 'install extensions from Configure' feature, which means it doesn't work on Dreamhost currently - see InstallingPluginsDreamhostErrors for details and workaround.

-- RichardDonkin - 11 Mar 2007

TWiki runs very, very slowly on my DH machine. So I decided to keep hosting my TWiki myself. Sometimes I have to wait more than 10 seconds for a view request to complete. It all looked great at first sight but turned out to be far too slow in the end. As anyone been able to get it running fast enough? I just looked at it again and it seems ok today... I guess it depends of your server workload at the time of the request.

-- StephaneLenclud - 12 Mar 2007

I never found any problems with performance, but I was running older TWiki versions and on a different shared server no doubt. I would log a support issue with Dreamhost, it's possible someone else on your shared server is tying up the CPU.

-- RichardDonkin - 12 Mar 2007

I have been with DH for several years now and always found the performance with TWiki "acceptable." That is, until recently. Over the past couple of months, the performance on my server ("gotcha") has degraded steadily until, at present, it is completely unacceptable. Based on a couple of client's complaints, I had to get a different hosting account and have moved those sites over. (The performance there is much better btw, with TWiki pages loading with scarcely a pause.)

So I'm a bit discouraged about DH. I basically like them: the package is very complete and the price is good. I've filed a request to do something about performance on my server or move me. We'll see...

-- LynnwoodBrown - 13 Mar 2007

Interesting - it's possible that with their one-click installs more people are running blog and wiki software than before, so average load on servers has gone up a lot... Or they are just overloading the servers. Worth also finding the dreamhost forums and commenting there.

-- RichardDonkin - 13 Mar 2007

LynnwoodBrown: by "...different hosting account..." you mean another account at DH or at some other hosting? Which one did you go for?

I think that's the very nature of shared hosting, you just can't rely on your server responsiveness. DH are probably victim of their success, too many customers == overloaded servers.

-- StephaneLenclud - 13 Mar 2007