When a view request is made for a non-existent web or topic, the HTTP status is returned as 200. It seems like it would be desirable to treat this as a 404 "page not found" error, so that automated tools such as spiders can treat this situation appropriately. Returning the 200 "ok" status will cause topics to remain indexed after they've been deleted or renamed.

Are there any drawbacks to returning the 404 status for web or topic not found?

-- EliMantel - 31 Dec 2001

From the spider's perspective a 404 is more appropriate, but from a Wiki user's perspective 200 "ok" is correct because it is just an invitation to create a topic.

Not sure how this could be done. Ideally a 404 or 200 should be returned based on the client type.

-- PeterThoeny - 31 Dec 2001

PeterThoeny said:

Not sure how this could be done. Ideally a 404 or 200 should be returned based on the client type.

I'm not sure it is a good idea since it may interfere with using curl or wget to work with a wiki site (e.g. bulk uploads or scanning for broken links, expired pages etc.) I really hate sites that change their function (as opposed to tailoring the output) based on the browser.

I may be totally wrong but I think I am reading the 1.1 cgi spec correctly, and we are using parsed headers to talk to the servers correct?

The CGI's create all of the header lines so that should be able to return a Status: header. The assumption I make is that TWiki can return a status 404 page that is identical in content and will work the same in a broswer as a status 200 page. The article http://www.plinko.net/404/custom.asp#ie describes how Internet Explorer will display the 404 page provided by the server rather than its internal page if the server page is > 512 bytes.

In all of the cases below, the only thing that changes from the current twiki response is the addition of the Status: 404 header.

• the web doesn't exist

Not a problem here. Web creation on demand just doesn't work 8-).

• A view URL was used

and

• the page doesn't exist.

this would be the case for a renamed page. When it existed somebody cached a view link to it. TWiki would still return the "Create the page or search" oops page so that people who want to create a new page can. This case can be distinguished from the create a non-existant page using the Jump To Page box because the jump to page uses the ?topic parameter. Direct page creation uses an edit link, so this rule doesn't apply.

Any other URL access returns code 200. This should provide all of the functionality to users since the generated 404 pages are the same as the current 200 pages, but the 404 wil stop robots from getting incorrect info for pages that used to exist.

-- JohnRouillard - 01 Jan 2002

I've hacked this into my site at http://twiki.cageyconsumer.com (which uses the December 2000 release). I tested it using IE and Netscape, and the most serious problem -- actually the only problem -- I noticed was that the page wasn't cached. The main effect of this is that if you change the contents of the search field, click on search, and then click on the browser back button, the search field will be reset to its original contents.

There are some other possible approaches to resolving this problem, such as setting header fields to control caching and to ask robots not to index the page. But I have a good deal more confidence that robots will pay attention to the status code than to a robot noindex request.

-- EliMantel - 03 Jan 2002