This allows you to %INCLUDE{}% sites that are virtually hosted, for example, twiki.org itself, where the site doesn't have it's own IP address, but the same IP is shared among many domain names. The URL is passed in the HTTP headers, and the web server uses that to return the correct pages for the correct site.

For example, you should see a page included between these two lines, but you don't:

Failed to include URL http://www.thoeny.com/peter/twiki/index.html Forbidden

• Replaced above link with http://www.thoeny.com/peter/twiki/index.html because www.geneticsound.com does no longer exist. Error is here because of missing proxy settings. -- PeterThoeny

That domain, geneticsound.com, is running on a non-unique IP, using an HTTP/1.1 virtual host.

Proper URL syntax also allows you to include restricted URLs, because it supports basic HTTP authentication, using the proper URL format of:

http://username:password@some.url.com/

Details on the HTTP/1.1 specification can be found here: http://www.w3.org/Protocols/

The changes modify TWiki.pm and Net.pm, and require MIME::Base64. I've provided the changes here both as replacement files for TWiki.pm and Net.pm, as well as unified diffs. And of course, it'd be super-swell if this made it into the main codebase.

-- VitoMiliano - 24 Feb 2002

See ProblemRenderingLocalCityTime - according to PeterThoeny the problem with the above INCLUDE is because it needs to be aware of proxies (SourceForge uses a proxy for outbound HTTP requests apparently).

Does anyone know where this sort of SourceForge setup info is documented? I had a look on sourceforge itself but it's hard to find amongst all the proxy projects...

-- RichardDonkin - 22 Mar 2002

can this issue be side-stepped by doing a Client-side include rather than a server side one?

-- SvenDowideit - 29 Mar 2002

It could, but part of the appeal of using TWiki is that just about everything dynamic is done server-side, and it doesn't depend on the client for anything other than standard HTML rendering. This is a mark of a well-designed site, in my opinion, and I'd like to see all parts of TWiki hold to this philosophy, since it allows the greatest number of users on the greatest number of platforms and clients to access and use every part of the site. Using client-side includes could cause problems with older or less feature-rich browsers.

I'm going to add outbound proxy support soon.

-- RyanFreebern - 10 Apr 2002

I've added outbound proxy support now. It was a simple process of just a few lines of changes to Net.pm -- the modified version is attached below, as well as a unified diff file suitable for patching the Dec. 1, 2001 release of Net.pm.

To use an outbound HTTP proxy with this, just add two new variables to TWikiPreferences:

• • Set PROXYHOST = localhost
  • Set PROXYPORT = 8080

(Obviously, replace "localhost" and "8080" with the correct values for your system.)

Bug reports are welcome.

-- RyanFreebern - 19 Apr 2002

Is fixed now and in TWikiAlphaRelease and TWiki.org. Vito and Ryan: Thanks for the contribs!

Note: I changed PROXY_HOST to PROXYHOST, PROXY_PORT to PROXYPORT to bring it in line with other TWikiPreferences variables.

It does not yet work here on TWiki.org because it seems like SF has a proxy. I filed a support request asking for the proxy host and port, https://sourceforge.net/tracker/?func=detail&aid=599745&group_id=1&atid=350001

-- PeterThoeny - 02 Aug 2002

I've updated Net.pm in CVS to improve its use of modules and fix MimeBase64NowRequired, which caused MacOS X problems:

• Changed the 'use Socket' to do a 'require / import' combination so this happens at run-time and only when the getUrl() sub is actually called. Also made minor changes to socket calls to make this work under use strict. I have tested this code in a separate test module and it connects OK to a web server, so it should work.
• Changed the 'use MIME::Base64' to do a require/import where it is actually needed.

Generally, 'use Module' should be avoided wherever possible, as it incurs a module load overhead for just about every use of TWiki, and also creates a mandatory installation requirement (see CPAN and CpanPerlModulesRequirement). You can just do the following, which is equivalent to use MIME::Base64, right where you need that module, subject to some tinkering for modules like Socket that create subroutines intended for use as C-style constants:

require MIME::Base64;
import MIME::Base64 ();

Perl will take care of not re-importing a module if you go through the same code path twice.

-- RichardDonkin - 30 Aug 2002

After a bit of experiementation on SF, I have concluded that the proxy is at http://proxy.sourceforge.net:8080

M. -- MartinCleaver - 21 Dec 2002

What specifically is still needed to bring this BeijingRelease feature status up from 0% documentation?

-- GrantBow - 09 Jan 2003

The only thing left is to make sure the TWikiSystemRequirements lists the correct requirements for all platforms (read Net.pm code to see if MimeBase64NowRequired for all platforms.

-- PeterThoeny - 09 Jan 2003

This is done now, right? If so, that's 100% on the documentation!

-- GrantBow - 28 Jan 2003

I've recently been put in the situation where I needed to use this code behind a proxy for various reasons. Given it broke (ProxiedIncludesBrokenImplementationBug) I think this page is now misleading:

• The code isn't HTTP/1.1 compliant - it's only HTTP/1.0 compliant - the request string should be changed to match.
• For HTTP/1.1 requests the Host: header must always be included - even to hosts with no virtual hosts.
• Whilst the Host: header doesn't exist in the HTTP/1.0 specs it was in common use - which is what led to the addition and requirement of specification in the 1.1 definition.
• A number of transparent proxies (including squid if traditionally configured) cannot determine the origin webserver to be contacted if a Host: header is missing - regardless of being a 1.1'ism. (Some proxies can - they look at the network layer)
• In the case of an explicit proxy the request line is completely broken. The request line should as a minimum be of the form:
  • METHOD proto://fully.qualified.host/URL HTTP/version

As a result I'd recommend doing the following:

1. Downgrade the request line to HTTP/1.0
2. Include the Host header in all requests
3. Send proxies "proper" proxy-HTTP requests - ie the absoluteURI format.

The patch I've produced specifically to fix 3 and also to implement 1. 2 looks like it should be happening but appears to run the risk of sending an empty Host: field (which may be a problem with some servers/proxies). The lack of support for sites on non-standard HTTP ports is also an issue.

-- MichaelSparks - 09 Jul 2003

Aha. At last I found it. The proxy for SF is above, now in bold (21Dec02).

-- MartinCleaver - 09 Jul 2003