Introduction To URL Rewriting

Many Web companies spend hours and hours agonizing over the best domain names for their clients. They try to find a domain name that is relevant and appropriate, sounds professional yet is distinctive, is easy to spell and remember and read over the phone, looks good on business cards and is available as a dot-com.
Or else they spend thousands of dollars to purchase the one they really want, which just happened to be registered by a forward-thinking and hard-to-find squatter in 1998.
They go through all that trouble with the domain name but neglect the rest of the URL, the element after the domain name. It, too, should be relevant, appropriate, professional, memorable, easy to spell and readable. And for the same reasons: to attract customers and improve in search ranking.
Fortunately, there is a technique called URL rewriting that can turn unsightly URLs into nice ones — with a lot less agony and expense than picking a good domain name. It enables you to fill out your URLs with friendly, readable keywords without affecting the underlying structure of your pages.
This article covers the following:

1. What is URL rewriting?
2. How can URL rewriting help your search rankings?
3. Examples of URL rewriting, including regular expressions, flags and conditionals;
4. URL rewriting in the wild, such as on Wikipedia, WordPress and shopping websites;
5. Creating friendly URLs;
6. Changing pages names and URLs;
7. Checklist and troubleshooting.

What Is URL Rewriting?

If you were writing a letter to your bank, you would probably open your word processor and create a file named something like lettertobank.doc. The file might sit in your Documents directory, with a full path like C:\Windows\users\julie\Documents\lettertobank.doc. One file path = one document.
Similarly, if you were creating a banking website, you might create a page named page1.html, upload it, and then point your browser to http://www.mybanksite.com/page1.html. One URL = one resource. In this case, the resource is a physical Web page, but it could be a page or product drawn from a CMS.
URL rewriting changes all that. It allows you to completely separate the URL from the resource. With URL rewriting, you could have http://www.mybanksite.com/aboutus.html taking the user to …/page1.html or to …/about-us/ or to …/about-this-website-and-me/ or to …/youll-never-find-out-about-me-hahaha-Xy2834/. Or to all of these. It’s a bit like shortcuts or symbolic links on your hard drive. One URL = one way to find a resource.
With URL rewriting, the URL and the resource that it leads to can be completely independent of each other. In practice, they’re usually not wholly independent: the URL usually contains some code or number or name that enables the CMS to look up the resource. But in theory, this is what URL rewriting provides: a complete separation.

How Does URL Rewriting Help?

Can you guess what this Web page sells?

http://www.diy.com/diy/jsp/bq/nav.jsp?action=detail&fh_secondid=11577676

B&Q went to all the trouble and expense of acquiring diy.com and implementing a stock controlled e-commerce website, but left its URLs indecipherable. If you guessed “brown guttering,” you might want to considering playing the lottery.
Even when you search directly for this “miniflow gutter brown” on Google UK, B&Q’s page comes up only seventh in the organic search results, below much smaller companies, such as a building supplier with a single outlet in Stirlingshire. B&Q has 300+ branches and so is probably much bigger in budget, size and exposure, so why is it not doing as well for this search term? Perhaps because the other search results have URLs like http://www.prof…co.uk/products/brown-miniflo-gutter-148/; that is, the URL itself contains the words in the search term.

Almost all of these results on Google have the search term in their URLs (highlighted in green). The one at the bottom does not.
Looking at the URL from B&Q, you would (probably correctly) assume that a file named nav.jsp within the directory /diy/jsp/bq/ is used to display products when given their ID number, 11577676 in this case. That is the resource intimately tied to this URL.
So, how would B&Q go about turning this into something more recognizable, like http://www.diy.com/products/miniflow-gutter-brown/11577676, without restructuring its whole website? The answer is URL rewriting.
Another way to look at URL rewriting is like a thin layer that sits on top of a website, translating human- and search-engine-friendly URLs into actual URLs. Doing it is easy because it requires hardly any changes to the website’s underlying structure — no moving files around or renaming things.
URL rewriting basically tells the Web server that
/products/miniflow-gutter-brown/11577676 should show the Web page at: /diy/jsp/bq/nav.jsp?action=detail&fh_secondid=11577676,
without the customer or search engine knowing about it.
Many factors (or “signals”), of course, determine the search ranking for a particular term, over 200 of them according to Google. But friendly and readable URLs are consistently ranked as one of the most important of those factors. They also help humans to quickly figure out what a page is about.
The next section describes how this is done.

How To Rewrite URLs

Whether you can implement URL rewriting on a website depends on the Web server. Apache usually comes with the URL rewriting module, mod_rewrite, already installed. The set-up is very common and is the basis for all of the examples in this article. ISAPI Rewrite is a similar module for Windows IIS but requires payment (about $100 US) and installation.

The Simplest Case

The simplest case of URL rewriting is to rename a single static Web page, and this is far easier than the B&Q example above. To use Apache’s URL rewriting function, you will need to create or edit the .htaccess file in your website’s document root (or, less commonly, in a subdirectory).
For instance, if you have a Web page about horses named Xu8JuefAtua.htm, you could add these lines to .htaccess:

1	RewriteEngine On

2	RewriteRule   horses.htm   Xu8JuefAtua.htm

Now, if you visit http://www.mywebsite.com/horses.htm, you’ll actually be shown the Web page Xu8JuefAtua.htm. Furthermore, your browser will remain at horses.htm, so visitors and search engines will never know that you originally gave the page such a cryptic name.

Introducing Regular Expressions

In URL rewriting, you need only match the path of the URL, not including the domain name or the first slash. The rule above essentially tells Apache that if the path contains horses.htm, then show the Web page Xu8JuefAtua.htm. This is slightly problematic, because you could also visit http://www.mywebsite.com/reallyfasthorses.html, and it would still work. So, what we really need is this:

1	RewriteEngine On

2	RewriteRule   ^horses.htm$   Xu8JuefAtua.htm

The ^horses.htm$ is not just a search string, but a regular expression, in which special characters — such as ^ . + * ? ^ ( ) [ ] { } and $ — have extra significance. The ^ matches the beginning of the URL’s path, and the $ matches the end. This says that the path must begin and end with horses.htm. So, only horses.htm will work, and not reallyfasthorses.htm or horses.html. This is important for search engines like Google, which can penalize what it views as duplicate content — identical pages that can be reached via multiple URLs.

Without File Endings

You can make this even better by ditching the file ending altogether, so that you can visit either http://www.mywebsite.com/horses or http://www.mywebsite.com/horses/:

1	RewriteEngine On

2	RewriteRule   ^horses/?$   Xu8JuefAtua.html  [NC]

The ? indicates that the preceding character is optional. So, in this case, the URL would work with or without the slash at the end. These would not be considered duplicate URLs by a search engine, but would help prevent confusion if people (or link checkers) accidentally added a slash. The stuff in brackets at the end of the rule gives Apache some further pointers. [NC] is a flag that means that the rule is case insensitive, so http://www.mywebsite.com/HoRsEs would also work.

Wikipedia Example

We can now look at a real-world example. Wikipedia appears to use URL rewriting, passing the title of the page to a PHP file. For instance…

http://en.wikipedia.org/wiki/Barack_obama

… is rewritten to:

http://en.wikipedia.org/w/index.php?title=Barack_obama

This could well be implemented with an .htaccess file, like so:

1	RewriteEngine On

2	#Look for the word "wiki" followed by a slash, and then the article title

3	RewriteRule   ^wiki/(.+)$   w/index.php?title=$1   [L]

The previous rule had /?, which meant zero or one slashes. If it had said /+, it would have meant one or more slashes, so even http://www.mywebsite.com/horses//// would have worked. In this rule, the dot (.) matches any character, so .+ matches one or more of any character — that is, essentially anything. And the parentheses — ( ) — ask Apache to remember what the .+ is. The rule above, then, tells Apache to look for wiki/ followed by one or more of any character and to remember what it is. This is remembered and then rewritten as $1. So, when the rewriting is finished, wiki/Barack_obama becomes w/index.php?title=Barack_obama
Thus, the page w/index.php is called, passing Barack_obama as a parameter. The w/index.php is probably a PHP page that runs a database lookup — like SELECT * FROM articles WHERE title='Barack obama' — and then outputs the HTML.

You can also view Wikipedia entries directly, without the URL rewriting.

Comments and Flags

The example above also introduced comments. Anything after a # is ignored by Apache, so it’s a good idea to explain your rewriting rules so that future generations can understand them. The [L] flag means that if this rule matches, Apache can stop now. Otherwise, Apache would continue applying subsequent rules, which is a powerful feature but unnecessary for all but the most complex rule sets.

Implementing the B&Q Example

The recommendation for B&Q above could be implemented with an .htaccess file, like so:

1	RewriteEngine On

2	#Look for the word "products" followed by slash, product title, slash, id number

3	RewriteRule  ^products/.*/([0-9]+)$   diy/jsp/bq/nav.jsp?action=detail&fh_secondid=$1 [NC,L]

Here, the .* matches zero or more of any character, so nothing or anything. And the [0-9] matches a single numerical digit, so [0-9]+ matches one or more numbers.
The next section covers a couple of more complex conditional examples. You can also read the Apache rewriting guide for much more information on all that URL rewriting has to offer.

Conditional Rewriting

URL rewriting can also include conditions and make use of environment variables. These two features make for an easy way to redirect requests from one domain alias to another. This is especially useful if a website changes its domain, from mywebsite.co.uk to mywebsite.com for example.

Domain Forwarding

Most domain registrars allow for domain forwarding, which redirects all requests from one domain to another domain, but which might send requests for www.mywebsite.co.uk/horses to the home page at www.mywebsite.com and not to www.mywebsite.com/horses. You can achieve this with URL rewriting instead:

1	RewriteEngine On

2	RewriteCond   %{HTTP_HOST}   !^www.mywebsite.com$         [NC]

3	RewriteRule   (.*)           http://www.mywebsite.com/$1  [L,R=301]

The second line in this example is a RewriteCond, rather than a RewriteRule. It is used to compare an Apache environment variable on the left (such as the host name in this case) with a regular expression on the right. Only if this condition is true will the rule on the next line be considered.
In this case, %{HTTP_HOST} represents www.mywebsite.co.uk, the host (i.e. domain) that the browser is trying to visit. The ! means “not.” This tells Apache, if the host does not begin and end with www.mywebsite.com, then remember and rewrite zero or more of any character to www.mywebsite.com/$1. This converts www.mywebsite.co.uk/anything-at-all to www.mywebsite.com/anything-at-all. And it will work for all other aliases as well, like www.mywebsite.biz/anything-at-all and mywebsite.com/anything-at-all.
The flag [R=301] is very important. It tells Apache to do a 301 (i.e. permanent) redirect. Apache will send the new URL back to the browser or search engine, and the browser or search engine will have to request it again. Unlike all of the examples above, the new URL will now appear in the browser’s location bar. And search engines will take note of the new URL and update their databases. [R] by itself is the same as [R=302] and signifies a temporary redirect.

File Existence and WordPress

Smashing Magazine runs on the popular blogging software WordPress. WordPress enables the author to choose their own URL, called a “slug.” Then, it automatically prepends the date, such as http://coding.smashingmagazine.com/2011/09/05/getting-started-with-the-paypal-api/. In your pre-URL rewriting days, you might have assumed that Smashing Magazine’s Web server was actually serving up a file located at …/2011/09/05/getting-started-with-the-paypal-api/index.html. In fact, WordPress uses URL rewriting extensively.

WordPress enables the author to choose their own URL for an article.
WordPress’ .htaccess file looks like this:

1	RewriteEngine On

2	RewriteBase /

3	RewriteCond %{REQUEST_FILENAME} !-f

4	RewriteCond %{REQUEST_FILENAME} !-d

5	RewriteRule . /index.php [L]

The -f means “this is a file” and -d means “this is a directory.” This tells Apache, if the requested file name is not a file, and the requested file name is not a directory, then rewrite everything (i.e. any path containing any character) to the page index.php. If you are requesting an existing image or the log-in page wp-login.php, then the rule is not triggered. But if you request anything else, like /2011/09/05/getting-started-with-the-paypal-api/, then the file index.php jumps into action.
Internally, index.php (probably) looks at the environment variable $_SERVER['REQUEST_URI'] and extracts the information that it needs to find out what it is looking for. This gives it even more flexibility than Apache’s rewrite rules and enables WordPress to mimic some very sophisticated URL rewriting rules. In fact, when administering a WordPress blog, you can go to Settings → Permalink on the left side, and choose the type of URL rewriting that you would like to mimic.

WordPress’ permalink settings, letting you choose the type of URL rewriting that you would like to mimic.

Rewriting Query Strings

If you are hired to recreate an existing website from scratch, you might use URL rewriting to redirect the 20 most popular URLs on the old website to the locations on the new website. This could involve redirecting things like prod.php?id=20 to products/great-product/2342, which itself gets redirected to the actual product page.
Apache’s RewriteRule applies only to the path in the URL, not to parameters like id=20. To do this type of rewriting, you will need to refer to the Apache environment variable %{QUERY_STRING}. This can be accomplished like so:

1	RewriteEngine On

2	RewriteCond   %{QUERY_STRING}           ^id=20$

3	RewriteRule   ^prod.php$             ^products/great-product/2342$      [L,R=301]

4	RewriteRule   ^products/(.*)/([0-9]+)$  ^productview.php?id=$1             [L]

In this example, the first RewriteRule triggers a permanent redirect from the old website’s URL to the new website’s URL. The second rule rewrites the new URL to the actual PHP page that displays the product.

Examples Of URL Rewriting On Shopping Websites

For complex content-managed websites, there is still the issue of how to map friendly URLs to underlying resources. The simple examples above did that mapping by hand, manually associating a URL like horses.htm with the file or resource Xu8JuefAtua.htm. Wikipedia looks up the resource based on the title, and WordPress applies some complex internal rule sets. But what if your data is more complex, with thousands of products in hundreds of categories? This section shows the approach that Amazon and many other shopping websites take.
If you’ve ever come across a URL like this on Amazon, http://www.amazon.co.uk/High-Voltage-AC-DC/dp/B00008AJL3, you might have assumed that Amazon’s website has a subdirectory named /High-Voltage-AC-DC/dp/ that contains a file named B00008AJL3.
This is very unlikely. You could try changing the name of the top-level “directory” and you would still arrive on the same page, http://www.amazon.co.uk/Test-Voltage-AC-DC/dp/B00008AJL3.
The bit at the end is what really matters. Looking down the page, you’ll see that B00008AJL3 is this AC/DC album’s ASIN (Amazon Standard Identification Number). If you change that, you’ll get a “Page not found” or an entirely different product: http://www.amazon.co.uk/High-Voltage-AC-DC/dp/B003BEZ7HI.
The /dp/ also matters. Changing this leads to a “Page not found.” So, the B00008AJL3 probably tells Amazon what to display, and the dp tells the website how to display it. This is URL rewriting in action, with the original URL possibly ending up getting rewritten to something like:
http://www.amazon.co.uk/displayproduct.php?asin=B00008AJL3.

Features of an Amazon URL

This introduces some important features of Amazon’s URLs that can be applied to any website with a complex set of resources. It shows that the URL can be automatically generated and can include up to three parts:

1. The wordsIn this case, the words are based on the album and artist, and all non-alphanumeric characters are replaced. So, the slash in AC/DC becomes a hyphen. This is the bit that helps humans and search engines.
2. An ID numberOr something that tells the website what to look up, such as B00008AJL3.
3. An identifierOr something that tells the website where to look for it and how to display it. If dp tells Amazon to look for a product, then somewhere along the line, it probably triggers a database statement such as SELECT * FROM products WHERE id='B00008AJL3'.

Other Shopping Examples

Many other shopping websites have URLs like this. In the list below, the ID number and (suspected) identifier are in bold:

• http://www.ebay.co.uk/itm/Ian-Rankin-Set-Darkness-Rebus-Novel-/140604842997
• http://www.kelkoo.com/c-138201-lighting/brand/caravan
• http://www.ciao.co.uk/Fridge_Freezers_5266430_3
• http://www.gumtree.com/p/for-sale/boys-bmx-bronx-blaze/97669042
• http://www.comet.co.uk/c/Televisions/LCD-Plasma-LED-TVs/1844

A significant benefit of this type of URL is that the actual words can be changed, as shown below. As long as the ID number stays the same, the URL will still work. So products can be renamed without breaking old links. More sophisticated websites (like Ciao above) will redirect the changed URL back to the real one and thus avoid creating the appearance of duplicate content (see below for more on this topic).

Websites that use URL rewriting are more flexible with their URLs — the words can change but the page will still be found.

Friendly URLs

Now you know how to map nice friendly URLs to their underlying Web pages, but how should you create those friendly URLs in the first place?
If we followed the current advice, we would separate words with hyphens rather than underscores and capitalize consistently. Lowercase might be preferable because most people search in lowercase. Punctuation such as dots and commas should also be turned into hyphens, otherwise they would get turned into things like %2C, which look ugly and might break the URL when copied and pasted. You might want to remove apostrophes and parentheses entirely for the same reason.
Whether to replace accented characters is debatable. URLs with accents (or any non-Roman characters) might look bad or break when rendered in a different character format. But replacing them with their non-accented equivalents might make the URLs harder for search engines to find (and even harder if replaced with hyphens). If your website is for a predominately French audience, then perhaps leave the French accents in. But substitute them if the French words are few and far between on a mainly English website.
This PHP function succinctly handles all of the above suggestions:

1	function GenerateUrl ($s) {

2	//Convert accented characters, and remove parentheses and apostrophes

3	$from = explode (',', "ç,æ,œ,á,é,í,ó,ú,à,è,ì,ò,ù,ä,ë,ï,ö,ü,ÿ,â,ê,î,ô,û,å,e,i,ø,u,(,),[,],'");

4	$to = explode (',', 'c,ae,oe,a,e,i,o,u,a,e,i,o,u,a,e,i,o,u,y,a,e,i,o,u,a,e,i,o,u,,,,,,');

5	//Do the replacements, and convert all other non-alphanumeric characters to spaces

6	$s = preg_replace ('~[^\w\d]+~', '-', str_replace ($from, $to, trim ($s)));

7	//Remove a - at the beginning or end and make lowercase

8	return strtolower (preg_replace ('/^-/', '', preg_replace ('/-$/', '', $s)));

9

}

This would generate URLs like this:

1	echo GenerateUrl ("Pâtisserie (Always FRESH!)"); //returns "patisserie-always-fresh"

Or, if you wanted a link to a $product variable to be pulled from a database:

1	$product = array ('title'=>'Great product', 'id'=>100);

2	echo '. GenerateUrl ($product['title']) . '/' . $product['id'] . '">';

3	echo $product['title'] . '';

Changing Page Names

Search engines generally ignore duplicate content (i.e. multiple pages with the same information). But if they think they are being manipulated, search engines will actively penalize the website, so avoid this where possible. Google recommends using 301 redirects to send users from old pages to new ones.
When a URL-rewritten page is renamed, the old URL and new URL should both still work. Furthermore, to avoid any risk of duplication, the old URL should automatically redirect to the new one, as WordPress does.
Doing this in PHP is relatively easy. The following function looks at the current URL, and if it’s not the same as the desired URL, it redirects the user:

1	function CheckUrl ($s) {

2	// Get the current URL without the query string, with the initial slash

3	$myurl = preg_replace ('/\?.*$/', '', $_SERVER['REQUEST_URI']);

4	//If it is not the same as the desired URL, then redirect

5	if ($myurl != "/$s") {Header ("Location: /$s", true, 301); exit;}

6

}

This would be used like so:

1	$producturl = GenerateUrl ($product['title']) . '/' . $product['id'];

2	CheckUrl ($producturl); //redirects the user if they are at the wrong place

If you would like to use this function, be sure to test it in your environment first and with your rewrite rules, to make sure that it does not cause any infinite redirects. This is what that would look like:

This is what happens when Google Chrome visits a page that redirects to itself.

Checklist And Troubleshooting

Use the following checklist to implement URL rewriting.

1. Check That It’s Supported

Not all Web servers support URL rewriting. If you put up your .htaccess file on one that doesn’t, it will be ignored or will throw up a “500 Internal Server Error.”

2. Plan Your Approach

Figure out what will get mapped to what, and how the correct information will still get found. Perhaps you want to introduce new URLs, like my-great-product/p/123, to replace your current product URLs, like product.php?id=123, and to substitute new-category/c/12 for category.php?id=12.

3. Create Your Rewrite Rules

Create an .htaccess file for your new rules. You can initially do this in a /testing/ subdirectory and using the [R] flag, so that you can see where things go:

1	RewriteEngine On

2	RewriteRule   ^.+/p/([0-9]+)   product.php?id=$1    [NC,L,R]

3	RewriteRule   ^.+/c/([0-9]+)   category.php?id=$1    [NC,L,R]

Now, if you visit www.mywebsite.com/testing/my-great-product/p/123, you should be sent to www.mywebsite.com/testing/product.php?id=123. You’ll get a “Page not found” because product.php is not in your /testing/ subdirectory, but at least you’ll know that your rules work. Once you’re satisfied, move the .htaccess file to your document root and remove the [R] flag. Now www.mywebsite.com/my-great-product/p/123 should work.

4. Check Your Pages

Test that your new URLs bring in all the correct images, CSS and JavaScript files. For example, the Web browser now believes that your Web page is named 123 in a directory named my-great-product/p/. If the HTML refers to a file named images/logo.jpg, then the Web browser would request the image from www.mywebsite.com/my-great-product/p/images/logo.jpg and would come up with a “File not found.”
You would need to also rewrite the image locations or make the references absolute (like ) or put a base href at the top of the of the page (). But if you do that, you would need to fully specify any internal links that begin with # or ? because they would now go to something like product.php#details.

5. Change Your URLs

Now find all references to your old URLs, and replace them with your new URLs, using a function such as GenerateUrl to consistently create the new URLs. This is the only step that might require looking deep into the underlying code of your website.

6. Automatically Redirect Your Old URLs

Now that the URL rewriting is in place, you probably want Google to forget about your old URLs and start using the new ones. That is, when a search result brings up product.php?id=20, you’d want the user to be visibly redirected to my-great-product/p/123, which would then be internally redirected back to product.php?id=20.
This is the reverse of what your URL rewriting already does. In fact, you could add another rule to .htaccess to achieve this, but if you get the rules in the wrong order, then the browser would go into a redirect loop.
Another approach is to do the first redirect in PHP, using something like the CheckUrl function above. This has the added advantage that if you rename the product, the old URL will immediately become invalid and redirect to the newest one.

7. Update and Resubmit Your Site Map

Make sure to carry through your new URLs to your site map, your product feeds and everywhere else they appear.

Conclusion

URL rewriting is a relatively quick and easy way to improve your website’s appeal to customers and search engines. We’ve tried to explain some real examples of URL rewriting and to provide the technical details for implementing it on your own website. Please leave any comments or suggestions below.

A Basic guide to HTAccess and few tips

Introduction to .htaccess..

This work in constant progress is some collected wisdom, stuff I've learned on the topic of .htaccess hacking, commands I've used successfully in the past, on a variety of server setups, and in most cases still do. You may have to tweak the examples some to get the desired result, though, and a reliable test server is a powerful ally, preferably one with a very similar setup to your "live" server. Okay, to begin..

..a win32 Apache mirror of corz.org     

.htaccess files are invisible

There's a good reason why you won't see .htaccess files on the web; almost every web server in the world is configured to ignore them, by default. Same goes for most operating systems. Mainly it's the dot "." at the start, you see?

If you don't  see, you'll need to disable your operating system's invisible file functions, or use a text editor that allows you to open hidden files, something like bbedit on the Mac platform. On windows, showing invisibles in explorer should allow any text editor to open them, and most decent editors to save them too**. Linux dudes know how to find them without any help.

that same folder, as seen from Mac OS X

In both images, the operating system has been instructed to display invisible files. ugly, but necessary sometimes. You will also need to instruct your ftp client to do the same.

By the way; the windows screencap is more recent than the mac one, moved files are likely being handled by my clever 404 script.

** even notepad can save files beginning with a dot, if you put double-quotes around the name when you save it; i.e.. ".htaccess". You can also use your ftp client to rename files beginning with a dot, even on your local filesystem; works great in FileZilla.

What are .htaccess files anyway?

Simply put, they are invisible plain text files where one can store server directives. Server directives are anything you might put in an Apache config file (httpd.conf) or even a php.ini**, but unlike those "master" directive files, these .htaccess directives apply only to the folder in which the .htaccess file resides, and all the folders inside.

This ability to plant .htaccess files in any directory of our site allows us to set up a finely-grained tree of server directives, each subfolder inheriting properties from its parent, whilst at the same time adding to, or over-riding certain directives with its own .htaccess file. For instance, you could use .htacces to enable indexes all over your site, and then deny indexing in only certain subdirectories, or deny index listings site-wide, and allow indexing in certain subdirectories. One line in the .htaccess file in your root and your whole site is altered. From here on, I'll probably refer to the main .htaccess in the root of your website as "the master .htaccess file", or "main" .htaccess file.

There's a small performance penalty for all this .htaccess file checking, but not noticeable, and you'll find most of the time it's just on and there's nothing you can do about it anyway, so let's make the most of it..

** Your main php.ini, that is, unless you are running under phpsuexec, in which case the directives would go inside individual  php.ini files

Is .htaccess enabled?

It's unusual, but possible that .htaccess is not enabled on your site. If you are hosting it yourself, it's easy enough to fix; open your httpd.conf in a text editor, and locate this section..

Your DocumentRoot may be different, of course..

# This should be changed to whatever you set DocumentRoot to.
#

#

..locate the line that reads..

AllowOverride None

..and change it to..

AllowOverride All

Restart Apache. Now .htaccess will work. You can also make this change inside a virtual host, which would normally be preferable.

If your site is hosted with someone else, check your control panel (Plesk. CPanel, etc.) to see if you can enable it there, and if not, contact your hosting admins. Perhaps they don't allow this. In which case, switch to a better web host.

What can I do with .htaccess files?

Almost any directive that you can put inside an httpd.conf file will also function perfectly inside an .htaccess file. Unsurprisingly, the most common use of .htaccess is to..

Control access..

.htaccess is most often used to restrict or deny access to individual files and folders. A typical example would be an "includes" folder. Your site's pages can call these included scripts all they like, but you don't want users accessing these files directly, over the web. In that case you would drop an .htaccess file in the includes folder with content something like this..

NO ENTRY!

# no one gets in here!
deny from all

which would deny ALL direct access to ANY files in that folder. You can be more specific with your conditions, for instance limiting access to a particular IP range, here's a handy top-level rule for a local test server..

NO ENTRY outside of the LAN!

# no nasty crackers in here!
order deny,allow
deny from all
allow from 192.168.0.0/24
# this would do the same thing..
#allow from 192.168.0

Generally these sorts of requests would bounce off your firewall anyway, but on a live server (like my dev mirror sometimes is) they become useful for filtering out undesirable IP blocks, known risks, lots of things. By the way, in case you hadn't spotted; lines beginning with "#" are ignored by Apache; handy for comments.

Sometimes, you will only want to ban one IP, perhaps some persistent robot that doesn't play by the rules..

post user agent every fifth request only. hmmm. ban IP..

# someone else giving the ruskies a bad name..
order allow,deny
deny from 83.222.23.219
allow from all

The usual rules for IP addresses apply, so you can use partial matches, ranges, and so on. Whatever, the user gets a 403 "access denied" error page in their client software (browser, usually), which certainly gets the message across. This is probably fine for most situations, but in part two I'll demonstrate some cooler ways to deny access, as well as how to deny those nasty web suckers, bad referrers, script kiddies and more.

Custom error documents..

I guess I should briefly mention that .htaccess is where most folk configure their error documents. Usually with sommething like this..

the usual method. the "err" folder (with the custom pages) is in the root

# custom error documents
ErrorDocument 401 /err/401.php
ErrorDocument 403 /err/403.php
ErrorDocument 404 /err/404.php
ErrorDocument 500 /err/500.php

You can also specify external URLs, though this can be problematic, and is best avoided. One quick and simple method is to specify the text in the directive itself, you can even use HTML (though there is probably a limit to how much HTML you can squeeze onto one line). Remember, for Apache 1; begin with a ", but DO NOT end with one. For Apache 2, you can put a second quote at the end, as normal.

measure twice, quote once..

# quick custom error "document"..
ErrorDocument 404 "

There is nothing here.. go away quickly!

Using a custom error document is a Very Good Idea, and will give you a second chance at your almost-lost visitors. I recommend you download mine. But then, I would.

Password protected directories..

The next most obvious use for our .htaccess files is to allow access to only specific users, or user groups, in other words; password protected folders. a simple authorisation mechanism might look something like this..

a simple sample .htaccess file for password protection:

AuthType Basic
AuthName "restricted area"
AuthUserFile /usr/local/var/www/html/.htpasses
require valid-user

You can use this same mechanism to limit only certain kinds of requests, too..

only valid users can POST in here, anyone can GET, PUT, etc:

AuthType Basic
AuthName "restricted area"
AuthUserFile /usr/local/var/www/html/.htpasses

 require valid-user

You can find loads of online examples of how to setup authorization using .htaccess, and so long as you have a real  user (or create one, in this case, 'jimmy') with a real  password (you will be prompted for this, twice) in a real  password file (the -c switch will create it)..

htpasswd -c /usr/local/var/www/html/.htpasses jimmy

..the above will work just fine. htpasswd is a tool that comes free with Apache, specifically for making and updating password files, check it out. The windows version is the same; only the file path needs to be changed; to wherever you want to put the password file.

Note: if the Apache bin/ folder isn't in your PATH, you will need to cd into that directory before performing the command. Also note: You can use forward and back-slashes interchangeably with Apache/php on Windows, so this would work just fine..

htpasswd -c c:/unix/usr/local/Apache2/conf/.htpasses jimmy

Relative paths are fine too; assuming you were inside the bin/ directory of our fictional Apache install, the following would do exactly the same as the above..

htpasswd -c ../conf/.htpasses jimmy

Naming the password file .htpasses is a habit from when I had to keep that file inside the web site itself, and as web servers are configured to ignore files beginning with .ht, they too, remain hidden. If you keep your password file outside the web root (a better idea), then you can call it whatever you like, but the .ht_something habit is a good one to keep, even inside the web tree, it is secure enough for our basic  purpose..

Once they are logged in, you can access the remote_user environmental variable, and do stuff with it..

the remote_user variable is now available..

RewriteEngine on
RewriteCond %{remote_user} !^$ [nc]
RewriteRule ^(.*)$ /users/%{remote_user}/$1

Which is a handy directive, utilizing mod_rewrite; a subject I delve into far more deeply, in part two.

Get better protection..

The authentication examples above assume that your web server supports "Basic" http authorisation, as far as I know they all do (it's in the Apache core). Trouble is, some browsers aren't sending password this way any more, personally I'm looking to php to cover my authorization needs. Basic auth works okay though, even if it isn't actually very secure - your password travels in plain text over the wire, not clever.

If you have php, and are looking for a more secure login facility, check out pajamas. It's free. If you are looking for a password-protected download facility (and much more, besides), check out my distro machine, also free.

500 error..

If you add something that the server doesn't understand or support, you will get a 500 error page, aka.. "the server did a boo-boo". Even directives that work perfectly on your test server at home may fail dramatically at your real site. In fact this is a great way to find out if .htaccess files are enabled on your site; create one, put some gibberish in it, and load a page in that folder, wait for the 500 error. if there isn't one, probably they are not enabled.

If they are, we need a way to safely do live-testing without bringing the whole site to a 500 standstill.

Fortunately, in much the same way as we used the tag above, we can create conditional directives, things which will only come into effect if certain conditions are true. The most useful of these is the "ifModule" condition, which goes something like this..

only if PHP is loaded, will this directive have any effect (switch the 4 for a 5 if using php5)

 php_value default_charset utf-8

..which placed in your master .htaccess file, that would set the default character encoding of your entire site to utf-8 (a good idea!), at least, anything output by PHP. If the PHP4^** module isn't running on the server, the above .htaccess directive will do exactly nothing; Apache just ignores it. As well as proofing us against knocking the server into 500 mode, this also makes our .htaccess directives that wee bit more portable. Of course, if your syntax is messed-up, no amount of if-module-ing is going to prevent a error of some kind, all the more reason to practice this stuff on a local test server.

** note: if you are using php5, you would obviously instead use .

Groovy things to do with .htaccess..

So far we've only scratched the surface. Aside from authorisation, the humble .htaccess file can be put to all kinds of uses. If you've ever had a look in my public archives you will have noticed that that the directories are fully browsable, just like in the old days before adult web hosts realized how to turn that feature off! A line like this..

bring back the directories!

Options +Indexes +MultiViews +FollowSymlinks

..will almost certainly turn it back on again. And if you have mod_autoindex.c installed on your server (probably, yes), you can get nice fancy indexing, too..

show me those files!

 IndexOptions FancyIndexing

..which, as well as being neater, allows users to click the titles and, for instance, order the listing by date, or file size, or whatever. It's all for free too, built-in to the server, we're just switching it on. You can control certain parameters too..

let's go all the way!

 IndexOptions FancyIndexing IconHeight=16 IconWidth=16

Other parameters you could add include..

NameWidth=30
DescriptionWidth=30
IconsAreLinks SuppressHTMLPreamble (handy!)

I'm not mentioning the "XHTML" parameter in Apache2, because it still isn't! Anyways, I've chucked one of my old fancy indexing .htaccess file onsite for you to have some fun with. Just add readme.html and away you go! note: these days I use a single header files for all  the indexes..

HeaderName /inc/header.html

.. and only drop in local "readme" files. Check out the example, and my public archives for more details.

custom directory index files

While I'm here, it's worth mentioning that .htaccess is where you can specify which files you want to use as your indexes, that is, if a user requests /foo/, Apache will serve up /foo/index.html, or whatever file you specify.

You can also specify multiple files, and Apache will look for each in order, and present the first one it finds. It's generally setup something like..

DirectoryIndex index.html index.php index.htm

It really is worth scouting around the Apache documentation, often you will find controls for things you imagined were uncontrollable, thereby creating new possibilities, better options for your website. My experience of the magic "LAMP" (Linux-Apache-MySQL-PHP) has been.. "If you can imagine that it can be done, it can be done". Swap "Linux" for any decent operating system, the "AMP" part runs on most of them.

Okay, so now we have nice fancy directories, and some of them password protected, if you don't watch out, you're site will get popular, and that means bandwidth..

Save bandwidth with .htaccess!

If you pay for your bandwidth, this wee line could save you hard cash..

save me hard cash! and help the internet!

 php_value zlib.output_compression 16386

All it does is enables PHP's built-in transparent zlib compression. This will half your bandwidth usage in one stroke, more than that, in fact. Of course it only works with data being output by the PHP module, but if you design your pages with this in mind, you can use php echo statements, or better yet, php "includes" for your plain html output and just compress everything! Remember, if you run phpsuexec, you'll need to put php directives in a local php.ini file, not .htaccess. See here for more details.

Hide and deny files..

Do you remember I mentioned that any file beginning with .ht is invisible? .."almost every web server in the world is configured to ignore them, by default" and that is, of course, because .ht_anything files generally have server directives and passwords and stuff in them, most  servers will have something like this in their main configuration..

Standard setting..

 Order allow,deny
 Deny from all
 Satisfy All

which instructs the server to deny access to any file beginning with .ht, effectively protecting our .htaccess and other files. The "." at the start prevents them being displayed in an index, and the .ht prevents them being accessed. This version..

ignore what you want

 Order allow,deny
 Deny from all
 Satisfy All

tells the server to deny access to *.log files. You can insert multiple file types into each rule, separating them with a pipe "|", and you can insert multiple blocks into your .htaccess file, too. I find it convenient to put all the files starting with a dot into one, and the files with denied extensions into another, something like this..

the whole lot

# deny all .htaccess, .DS_Store $hî†é and ._* (resource fork) files

 Order allow,deny
 Deny from all
 Satisfy All


# deny access to all .log and .comment files

 Order allow,deny
 Deny from all
 Satisfy All

would cover all ._* resource fork files, .DS_Store files (which the Mac Finder creates all over the place) *.log files, *.comment files and of course, our .ht* files. You can add whatever file types you need to protect from direct access. I think it's clear now why the file is called ".htaccess".

These days, using is preferred over , mainly because you can use regular expression in the conditions (very handy), produce clean, more readable code. Here's an example. which I use for my php-generated style sheets..

parse file.css and file.style with the php machine..

# handler for phpsuexec..

 SetHandler application/x-httpd-php

Any files with a *.css or *.style extension will now be handled by php, rather than simply served up by Apache. And because you can use regexp, you could do stuff like , which is handy. Any statements you come across can be advantageously replaced by statements. Good to know.

More stuff..

At the end of my .htaccess files, there always seems to be a section of "stuff"; miscellaneous commands, mainly php flags and switches; so it seems logical to finish up the page with a wee selection of those..

php flags, switches and other stuff..

# let's enable php (non-cgi, aka. 'module') for EVERYTHING..'
AddType application/x-httpd-php5 .htm .html .php .blog .comment .inc

# better yet..
AddHandler php5-script .php

# legacy php4 version..'
AddType application/x-httpd-php .htm .html .php .blog .comment .inc

# don't even think about setting this to 'on'
php_value register_globals off

# no session id's in the URL PULEEZE!
php_value session.use_trans_sid 0
# should be the same as..
php_flag session.use_trans_sid off
# using both should also work fine!

# php error logs..
php_flag display_errors off
php_flag log_errors on
php_value track_errors on
php_value error_log /home/cor/errors/phperr.log

# if you like to collect interesting php system shell access and web hack scripts
# get yourself a SECURE upload facility, and just let the script-kiddies come …
# in no time you will have a huge selection of fascinating code. If you want folk to
# also upload zips and stuff, you might want to increase the upload capacities..
php_value upload_max_filesize 12M
php_value post_max_size 12M

# php 5 only, afaik. handy when your server isn't where YOU are.
php_value date.timezone Europe/Aberdeen
# actually, Europe/Aberdeen isn't a valid php timezone, so that won't work.
# I recommend you check the php manual for this function, because many crazy places ARE!

Note: For most of the flags I've tested, you can use on/off and true/false interchangeably, as well as 0/1, also php_value and php_flag can be switched around while things continue to work as expected! I guess, logically, booleans should always be php_flag, and values, php_value; but suffice to say, if some php erm, directive  isn't working, these would all be good things to fiddle with!

Of course, the php manual explains all. The bottom line is; both will work fine, but if you use the wrong type in .htaccess, say, set a php_flag using php_value, a php ini_get() command, for instance, would return true, even though you had set the value to off, because it reads off value as a string, which of course evaluates to not-zero, i.e. 1, or "true". If you don't rely on get_ini(), or similar, it's not a problem, though clearly it's better to get it right from the start. By the way; one of the values above is incorrectly set. Did you spot it?

Most php settings, you can override inside your actual scripts, but I do find it handy to be able to set defaults for a folder, or an entire site, using .htaccess.