Mod_rewrite trouble: RewriteEngine Off seemingly not working for subdirectories under a rewritten directory

It took me hours and hours of swearing at the screen before I finally found the solution for a problem I was facing with mod_rewrite. The situation was the following:

I was applying pretty urls to the website of a customer of mine using mod_rewrite. The rewrite rules were identical to the Zend htaccess file, which looks like this:

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule !\.(js|ico|gif|jpg|png|css)$ /index.php

This ruleset basically says: Internally redirect every request to a URL that is

  • not an existing file and
  • not an existing directory and
  • not a typical html asset

to index.php in the root of the website.
This htaccess file approach is also automatically in effect inside existing subdirectories but that’s usually easilly overcome by having a htaccess file in there that says:

RewriteEngine Off

But THAT is where things went wrong this time. When I requested a file in both the root directory as well as this subdirectory, I could not ever trigger a 404. Also not when requesting a non-existent .jpg file which, as you can see in the RewriteRule, should not trigger mod_rewrite.

After a lot of searching I found 1 website that mentioned the cause of my trouble. The server the website was hosted on was a DirectAdmin powered webserver that has custom error documents enabled which I deleted from the filesystem somewhere in the past.

What I needed to do is add the following line to the top of the htaccess file in the root of the website:

ErrorDocument 404 "The thing you are looking for does not exist on this planet... Go away."

This solved the whole problem. Of course the message can be whatever you deem fit for your visitors.

I figured out that I had similar trouble as mentioned on that other website by having the redirect target be some other url, followed by %{REQUEST_FILENAME} like so:

RewriteRule .* http://www.google.com/%{REQUEST_FILENAME} [NE,L]

Hope this helps.

Menno

Mobile device recognition

The challenge

As a webdeveloper for a company that develops a very popular application for a wide range of mobile phones (Symbian, iPhone, MIDP and Windows Mobile) I have to deal with several thousands of different mobile browsers hitting our download portal. From a hosting perspective this is not much of a challenge because Mobile websites tend to use a minimal amount of bandwidth.

Challenge #1 is that we have to be able to serve the user a binary that is suited for his specific mobile device almost without requiring him to do more than one single click. As soon as more clicks are needed about 30% of the visitors leaves again so it’s critical that the devices are immediately identified.

Challenge #2 is to serve the mobile device with a webpage that is optimised for his mobile browser. Serving XHTML rather than WML or maybe even using a bit of javascript here and there, not to speak of device specific bugs we’d like to work around.

In the past we only had support for a dozen or so devices, but soon that grew to more than a hundred. Right now we’re even over 900 supported devices and a even larger bunch of devices we have no specific version for but that I DO like to recognize, merely for the face that a user will understand a message like “Sorry, but your “Strawberry 69” is not yet supported”.
Clear message. No time wasted searching for a non-existent binary.

“What a wonderfull world” you might think as a potential user and you would be right!
However, there’s also a whole batch of webdevelopers out there who actually have to make this recognition work and who do lack the funds and enthousiasm to buy all devices in the world so they can “fingerprint” them for automated recognition.

Here’s the good news: There is a way for them to partially overcome these expenses by using a certain type of databases called “Device Description Repositories“.
As far as I am aware of there’s currently only 1 open source database like that which is called WURFL and in my experience there’s a lot of devices not recognised.

The plan

For this reason I am planning to start a project in the form of an API which should make device detection a breeze. Simply call a URL, post the http-headers received from the visiting browser and get your answer in a easy to parse format. The W3C consortium has actually created the requirements for such an API and also a came up with a simple API, but the API I have in mind should in it’s simplest form be pretty different from this one.

My first goal would be to be able to reach a 95% accuracy detecting only the brand and model of the device, but soon after that it would be easy to start supplying other technical information about the device like screen resolution, operating system, javascript abilities etc.

The API should be free to use and the data collection would be done by storing and analyzing the incoming data. The more sites use it, the more input will be found. Of course it should also be possible for third parties to manually contribute to the database.

Of course this project currently only exists in my mind but I would LOVE to get some input about this from potential users and people in the business. The name I have in mind is Sulfur which is no abbreviation but just a word I happen to like :-)