An (Almost) Completely Native Windows Installation

See also: TWiki windows installer with apache&perl(4.2.3), TWiki VMware Virtual Machine(4.2.3) with auto updating TWiki & plugins , TWiki VMware Virtual Machine(4.0.4) and TWikiForWindowsPersonal for easy installs on Windows

This topic describes the installation of a TWiki platform on a Windows system without resorting to Cygwin. In fact, Cygwin was used on the first installation, but I thought it would be fun to use MS ServicesForUnix or Gnuwin32 and check if performance could be improved. As it turned out, using Cygwin may be a better solution, and the way to go if it is an option for you.

The topic title may not be the best, actually; indeed, I label as 'native' any piece of code engineered and built for win32, without compatibility layers. Apache and AS Perl fall into this category, though IIS would have been more buy-in oriented.

You might also have a look at IndigoPerlCookbook, that describes a slightly different approach.

The Deployment Environment

• OS: Windows 2003 SP1
• Web Server: Apache 2.2.0-ssl (from http://www.apachelounge.com, with mod_perl-2.0.3-2.2.0)
• Perl: AS Perl v5.8.7
• Grep: Gnuwin32 (from http://gnuwin32.sf.net, without RCS, as their port is broken)
• RCS: embedded RCSlite

Requirements

Just like the environment described above, the requirements are bound to be similar across many Windows-based settings.

• Integration with existing system (in the general sense)
• User-friendly interface, easy to learn and use

Indeed, TWiki itself readily addresses the two concerns: it uses a common web browser as an interface, and only the admin group needs to worry with technical (read deployment) details. Users can start writing right away. Naturally, the admin group still needs to cater for:

• smooth operation in InternetExplorer
• transparent user authentication, i.e., skip registration as prerequisite for editing
• avoid the need for specific user formation (let users find out about 'advanced' features)
• WYSIWYG editing of topics

This last requirement will aid in hooking users coming from Office (Word, PowerPoint, etc.) environments. Even though content/style separation is a very healthy phylosophy - one that actually lends itself to style guideline enforcement - a successful system must be flexible, and provide options. LaTeX-fans will welcome the TWiki shorthand, and PowerPoint people won't have to adjust to a new denkweise.

First Steps

As you read through, keep in mind that there are many ways to do the same thing; my description merely reflects my tastes (use the edit button and add content, if you wish!).

1. install Apache in your system
   • Keep the path short, and without spaces. It helps keep entropy low;
   • I registered the service later, through the command line - it provides for extra parameters, which may be helpful;
   • edit httpd.conf, possibly replicating the structure which ships in recent builds: a central file to control defaults, module loading and related, and included files with VirtualHost definitions for each site. You can even go one step further and separate the module loading and configuration.
   • follow the installation instructions regarding Apache. There's also an example httpd.conf shipped which you can use as a VirtualHost definition.

Apache should now be running, though without a couple important tweaks. Or it may not. I'm not a very experienced Apache person, so I came across these errors as I opened http://localhost/:

• Error 401: Denied - permissions are wrong in Apache. Take another look at your site.conf, httpd.conf or any .htaccess you may have, and RTFM (I actually forgot about the solutions as soon as the problem was resolved... sorry).
• Error 505: Internal Server Error - the shebang line is wrong in the perl scripts. Don't worry, this one was intentional - it's important to get acquainted with this error.

Check the SupportWeb for extra help, there's plenty of material on Apache.

1. install AS Perl
   • again, keep paths short and without spaces or non-ascii characters
   • install the required packages: open a console, and issue

     set http_proxy=http://proxy.site.com:port

     if you're behind a proxy. Call ppm search to find the package name (some don't map directly to CPAN names), and ppm install package to install each.
   • find rewriteshebang.pl inside %Twiki%\tools, and run it using perl rewriteshebang.pl. Give the full path to your perl.exe binary, without caring for the message saying it can't be found.
   • the installer should have added the perl binary to your system PATH. It's not compulsory, since Apache will search for it using the shebang line, though it may avoid problems later on.

Checkpoint 1

Open http://localhost/bin/configure (change the URL accordingly). Does it work? Good! Now read through the details, without worrying too much about errors. These will be addressed in the next step.

1. install Gnuwin32

This is a very interesting suite of software. If you add the binary directory to the path, you can even profit from a better console. Assuming you got the installer/downloader and built the tree in a nice path, short and without spaces, you can proceed to changing the Store Settings in the configure script.

• • replace the existing lines for FGREP and EGREP with

    D:\fully_native_b2\gnuwin32\bin\grep.exe %CS{|-i}% %DET{|-l}% -H -- %TOKEN|U% %FILES|F%

    ; I got them somewhere in Codev, actually
  • while you're at it, change the revision setting to RCSlite. As mentioned before, Gnuwin32's port is buggy and unsupported, but there's an official build at the RCS homepage. I haven't had the time to investigate on the proper settings, though (it seems the Windows environment requires slightly different flags).
  • if you're doing transparent authentication, you may leave the remaining settings untouched. For this description we still need to select templateLogin and allowLoginName, as they'll be handy for registration. You might change the password hash to sha1, as indicated in the installation instructions. They won't be used with our authentication, though.

Checkpoint 2

The configure script should be error-free by now, even though warnings will be showing. Check them to see if you're comfortable with them.

Small, but relevant tasks

Some things need to be done before the TWiki is ready for users. The first one is setting the smtp server details in the configure script, so that email can be generated. Then, you need to edit the %WIKIWEBMASTER% variable to a proper email address: if you leave admin@examplePLEASENOSPAM.com the mail relay might refuse to let messages through.

You could do some additional tasks now, like enabling the WYSIWYG plugin. Bear in mind though, that every plugin adds extra entropy to the installation, and it may be better to make sure the core TWiki is running flawlessly before moving on to the extras.

Performance and Authentication

mod_auth_sspi

If you're happy with the installation, now it's the time to load the modules. This TWiki was deployed in an windows lan, where we wanted to make use of NTLM auth, so we passed user identification (authentication, etc.) to Apache, or more correctly the auth_sspi module, found at http://www.gknw.de/phpbb/viewtopic.php?t=128. Using the standard NT credentials, which IE provides to the module, users are transparently identified and authorized. Right now, anyone without credentials would be denied access to the site - something which happens using browsers other than IE.

An issue with this solution is the fact that a user will still have to register in the TWiki sense, if he/she wants to benefit from the personal page. While a registered user will see 'FirstName'-'LastName' and a greeting, someone absent from the user list will see 'DOMAIN\ntusername?' - very annoying, because the backslash will confuse TWiki. LDAP could be a solution, if you change the policy to force every user to register before using TWiki. I had trouble finding out who cared for the LDAP server, and ended up scrapping its use. Nevertheless, I'm looking into an automated registering process that would perform a lookup for every unregistered user, create a user page and fill in the userform values with LDAP information.

mod_perl

With a solid TWiki installation, it's time to boost the performance: plug in the Apache mod_perl. This must come last - for example, if you use configure with mod_perl turned on, you'll see TWiki complaining bitterly about some error, or other erratic behaviour. mod_perl stores some information during the session, and won't be very keen on changes. Anyway, you'll notice a great increase in performance after plugging it in.

Keep in mind that you're limited to a single TWiki installation if you use mod_perl: every TWiki would share the same perl environment, which is something definitely uninteresting.

Checkpoint 3

The Twiki should be up and running. The best way to check is to run the important things:

• edit, which will require Perl and RCS;
• search, which uses grep;
• Registration, which should come up without password field, but displaying an already filled in LoginName;
• the URL you'll be giving your users, to make sure the redirect is working properly.

If you have setup a TWiki using this installation procedure and found it easy to follow please tag it as easy_install.

Windows Issues

Windows, although a fully-fledged desktop/server system, lacks some fundamental tools which are commonplace in Unix environments. The most conspicuous is the absence of a proper console/shell interface, with its commands and scripting support. Usage of Cygwin does help hard-headed people like myself 'feel at home' - well, sort of. Other advantages include:

• a CPAN Perl distribution, which allows compilation of packages in case of necessity;
• no change in shebang lines, and less work in updates therefore;
• RCSWrap instead of RCSlite

Arguments against Cygwin are mostly ideological, then, and those which are not rather irrelevant:

• extra complexity through the inclusion of a linux/windows compatibility layer (argueable)
• more disk space needed
• something else to manage, either the updates or security issues
• confusion with forward/backslashes (a personal issue)

-- Contributors: RenatoSerodio

Discussion

-- RenatoSerodio - 03 Apr 2006

Would there be anything in hinderance to simply zipping up the installed directories for the components mentioned here (the complete TWiki installation incl. environment), ready for distribution? That could perhaps make it more "plugin"-like to install TWiki at Windows 2003 Server?

-- SteffenPoulsen - 28 Apr 2006

That's a good idea.. with such a backup, one could cater for burned up disks and other problems.

Zipping everything together is possible, if one doesn't use installers that may pre-edit some config files to fit the system (saw something like that in Apache). As far as I know, AS Perl is the only piece that writes to the registry, and even that can be done without. Naturally, one may have to edit the environment variables, or write a script that does it. Then it's all about going throught the Apache, configure, twiki\bin\LocalLib.cfg and twiki\lib\LocalSite.cfg edit cycle.

One problem remains, unfortunately: such a packaging is probably only acceptable for internal/personal use, as there may be some license infringements if you bundle these blocks together. And then, although each piece works fine by itself, it's probably best to have control over what's happening at each step.

-- RenatoSerodio - 28 Apr 2006

The best way to re-use and help stablise such a solution is to contribute to TWikiInstallerWindowsContrib

-- MartinCleaver - 28 Apr 2006

One interesting alternative to "Windows-native" installations is to download and install TWiki as an 'appliance' running on top of Windows. Once downloaded, you can be up and running in about 10 minutes. See TWikiVMDebianStable.

-- RichardDonkin - 19 May 2006

My Windows Native installation has finally broke in an uninteresting way. I've reached 187 total topics (that seems really small to me) within a web. The Apache (2.2.4) error.log is showing "failed" for the fgrep and egrep win32 commands I've been using. I replaced them with the grep config noted above using the GNUWIN32 binaries and received the same error. At the command line (cmd.exe), all the commands (grep, fgrep and egrep) work fine with "star dot txt" as input. The patterns searched for by ActionTrackerPlugin and HierarchicalNavigation are found; but, not within the TWiki/Apache environment. I suspected a problem with the command line being too long. I removed about a third of the topics and ActionTrackerPlugin (actionsearch) began to work again. Still no joy on HierarchicalNavigation. Its almost enough to make me want to attempt a cygwin installation of everything just for comparison purposes. Silly me, I'm prototyping on windows for a final linux install. Maybe I should have went the cygwin route in the first place.

-- DewayneVanHoozer - 18 Jun 2007

In a web server TWiki environment the paths are longer than at the command line because TWiki is using absolute paths, so the directory information is preceded for every single file: It is not "star dot txt", but more like /some/very/long/path/to/twiki/data/*.txt.

On Windows, the problem can not be finally solved, but there are two ways to improve the situation:

1. Install your TWiki in a "short" path, e.g. D:\TWiki.
2. Reduce the number of file names which TWiki passes to every grep command. This needs a one-line hack in the code and would depend on your actual TWiki version. On a somewhat older TWiki VM I found the line:

        /home/httpd/twiki/lib/TWiki/Store/RcsFile.pm:341:    my $maxTopicsInSet = 512; # max number of topics for a grep call
        

Good luck!

-- HaraldJoerg - 19 Jun 2007

Dewayne, if you are prototyping a future Linux install, you really should consider installing TWikiVMDebianStable on Windows - it's a complete pre-installed Linux-based TWiki that is hosted on VMware on Windows. Most of the setup on this environment would be very similar to the final Linux setup, so any problems you have on the VM TWiki would also be relevant to the ultimate Linux server.

Failing that, Harald's suggestions are good, and quite simple to implement. This was also discussed (in a Linux/Unix context though) in ArgumentListTooLong - Cygwin would help you avoid this too, partly through larger arg list sizes, and partly through use of xargs.

-- RichardDonkin - 19 Jun 2007

Thanks for the best wishes. TWiki version 4.1.2 does not seem to have a simular maxTopicsInSet feature. I've also investigated NativeSearch; but, I am unable to link PCRE in my current MinGW environment. For now I've decided to push forward with the deployment to the linux servers. I'll get back to the windows install in a few weeks.

-- DewayneVanHoozer - 19 Jun 2007

Thanks RichardDonkin. I will look into TWikiVMDebianStable. I'm wondering if replacing the grep dependencies with glimpse wouldn't be an overall improvement. This idea may be fodder for another topic.

-- DewayneVanHoozer - 19 Jun 2007

I should note that, after a couple of oddities with windows and cygwin setups (like the grep arglist, the 512 has moved to lib/TWiki/Store/SearchAlgorithms/Forking.pm in SVN TWiki), I myself have moved to TWikiVMDebianStable quite a while ago. The only downside is the installation and memory footprint, and yet another system to maintain, third perl besides Activestate and cygwin perl. But I have never missed the windows installation since then.

-- HaraldJoerg - 20 Jun 2007

Glimpse seems to be a search tool - there are quite a few plugins for search tools for TWiki, see Plugins and in particular Google:twiki+plucene.

Replacing grep with glimpse is unlikely because it would make TWiki installation even more complex, though a search tool is better for large installations of TWiki.

I wrote the original WindowsInstallCookbook, but as Harald's pointed out there are some oddities with CygWin and Windows TWiki installations generally. I also had real trouble getting some TWiki Sandbox (security) code working on ActivePerl a while back, so despite having done a lot with TWiki on Windows, I now recommend TWikiVMDebianStable.

While the TWiki VM approach does mean another system to maintain, in practice there is very little to do. The only real drawback is that it uses an older TWiki version currently (4.0.4) so it's best to do a TWiki upgrade as soon as you install it - but that's a lot easier than doing a full install, and not necessary for people who are just exploring the idea of Wikis and what they can do.

-- RichardDonkin - 22 Jun 2007

I was able to get around the command line length problem by switching to PurePerlBackendForSearch. I like the VM approach much better.

-- DewayneVanHoozer - 25 Jun 2007