301 Redirect Guide, Problems And Solutions
25, Jun - Written by Ricardo Figueiredo in 301 Redirects
A good friend of mine just shared a blog post from SEOmoz with me a few mornings ago. I don’t keep up too much with SEOmoz these days, so it’s good to have someone point out when they write something that is more relevant to what I do – no offense SEOmoz, you’re cool 
Last week there was a pretty interesting blog post at SEOmoz, written by Everett Sizemore, worth checking out if you are about to embark in a website redesign, relaunch, or any kind of work that will result in a change of URL structure on your site.
Sizemore’s post inspired me to breakdown what he wrote about, and add some of my own experience to the mix. While SEOmoz’s audience are mainly composed of more savvy individuals such as myself, my blog targets business owners and people who really need to just understand what the heck we’re talking about. So this is for you Mr. Business Man!
To dumb it down, if you’re changing your site in a way that your URLs will change, -THIS IS IMPORTANT- you should read this whole post. Here’s an example of what I’m talking about:
- Your contact page is going from http://www.example.com/contact.html to http://www.example.com/contact-us/
- This applies to when the URL structure is changing, as exemplified above, as well as when the actual domain name is changing (going from www.exampledomain.com to www.newdomain.com for example).
If this is happening to your site, chances are most or all of your URLs are changing too. Let’s start with the basics:
Image Credit - Volacci
301 Redirects – What, Why & How?
Redirects are commands that we give to the search engines (and other entities) when we need to, well, redirect them elsewhere. Search engines are robots, and although they are really smart, their algorithms don’t always “get” the semantics of a website. Take this scenario as an example:
You are a lawyer, and you have a really old website for your law firm. It has some outdated content, some stuff doesn’t work right, and let’s be honest, it’s ugly. So you hire a nice chap like myself, or another cool dude or dudette to help you out. We work together and design a beautiful, simple, user friendly, and up to date website that truly represents your law firm. In this process, we restructure the content, create search engine friendly URLs, etc. When we are ready to “turn off” the old site, and “turn on” the new one, we have one crucial step to take in order to not lose any of the current keyword rankings and authority your old site has gained over the years (it doesn’t matter if you ever put effort on your old site or not), and that is setting up 301 redirects.
There are different types of redirects, and we’re not going to cover all of them here because all you need to know in order to accomplish the task of successfully transferring rankings and authority from your old site to your new site revolves around 301 redirects (a.k.a. permanent redirects).
The redirect commands are written in a file named .htaccess, and that file lives in the root folder of your site. That is if you’re in a Linux server. If your site is hosted on a .Net (Windows/ASP) server, then kindly close your browser now and get the hell out of my face. (just kidding, but really, that sucks for you)
There are many tools available for free to help you build 301 redirects into a .htaccess file. Having said that, if you are reading this post you probably don’t know a whole lot about this stuff, so I wouldn’t recommend you messing with this because it’s a pretty easy way to bring your whole site down and mess things up. If I were you, I’d get in touch with me for help, but if you want to try your luck, here’s where you can go to learn how to build this.
301 Redirects – The Problems
Let’s say you got in touch with me, and we were able to help you get the redirects setup prior to launching your new site. Or maybe you did it all by yourself? Now what?
Now we have to do a few tests to make sure everything checks out, and that the redirects are working as they should. Once all is confirmed, we push the site live (since this post is about the redirects, I’m ignoring all of the other stuff you need to do prior to launching your new site).
Okay, your site launched, everything seems to be working (knock – knock – knock on wood). We check the redirects by trying to access some old URLs and notice how we’re being redirected to the new locations. For example:
- Our old contact page was at http://www.example.com/contact.html. So we type that into our browser and we are immediately redirected to the new contact page, at http://www.example.com/contact-us/.
We even ping the old URL to see the server response and ensure we have a 301 redirect being picked up.
Phew! Good work everyone, we did everything right so the search engines will have no issues and everything will work splendidly right? NOPE! Well, maybe yes, maybe not. We just don’t know how the search engines will react, so we watch, we monitor, we act paranoid for a couple of weeks and we make sure everything is checking out.
If all steps were taken correctly, chances are that over the course of a few weeks, the search engines will “learn” the new site with the assistance of all of our redirects and on page optimization, and all of our rankings will stabilize. Expect your rankings to fluctuate after the launch, regardless of how well you think the redirects are setup.
Houston, we got a problem!
Let’s say that a week goes by, two weeks go by, and you are just watching more and more of your rankings disappearing, and things don’t seem to be going well at all. The first thing you do is double and triple check all of the work, is anything missing? Fix anything that may not have been done correctly. If everything seems like it was done correctly, don’t go crazy. You could have done the most perfect job with your redirects and still run into problems.
This takes us back to what the SEOmoz post was about.
Remember, everything we are doing is geared towards making the process of re-learning the site easier to the robots that are run by the search engine’s algorithms. Sometimes, those robots do things that are unpredictable, or have faults that cause problems for us, or even work “too well”.
The case study presented by Mr. Sizemore shows a site relaunch project where everything was done to the T, and yet, they saw a tremendous loss after the site launch. So what happened?
Although this is rare, it could happen to your site. In this instance, Google indexed the new URLs of the new site right away, but continued to have the old URLs in their index. The problem is that due to a lack of an entry point into the old URLs (even the sitemap.xml had been updated with the new URLs), they had not be re-cached since the launch of the new site. While Google indexed the new pages with ease, it didn’t know what to do with the old URLs in its index, and the 301 redirects worked, but it was almost completely bypassed by the search engines.
Mr. Sizemore, being the smart dude that he is, turned the old linking structure back on to allow an entry points to the old URLs. This let the crawlers get back to the old URLs, and be redirected to the new ones. Shortly afterwards, the problem was reversed.
Image Credit - SEOmoz
This is an example of a rare event that can happen when you’re messing with 301 redirects. One of many things that can go wrong.
I hope you enjoyed this post, and please feel free to let me know if you want to see anything added through the comments below.
About Ricardo Figueiredo
Ricardo Figueiredo is the co-founder of Elevated Search, a boutique firm of SEO experts based out of San Diego. Ricardo has over 8 years of experience SEO project management specializing in on-page optimization, link building, and local SEO and conversion optimization..
June 26th, 2012 on 8:37 pm
I Have this exact same issue now, been just over 6 weeks and site is in Limbo, the original 301 (which was 1 site to a new site with 11k of pages) went seamlessly, all the new URL’s were indexed within 48 however still have all the old ones too, 6 weeks on traffic down at least 80-85%
When you say turn on the old linking structure do you mean add the original site again files again to the server ?
If so ill give this a go, i did try and add the old sitemap back but that did not work
June 26th, 2012 on 10:18 pm
@Paul
Yes reactive at your old files again on the server so Google can ping them. This will help them understand the move.
I know it sounds crazy but I would double check the redirects are 301s. I have had programmers say they 301ed the urls when the 302ed them.
Let us know how it goes.
September 13th, 2012 on 9:15 pm
Hi Steve
Thought i would update you, On my situation, i totally forgot I posted here and just stumbled across again.
14 weeks on im still in limbo, my 301 went like a charm perfect, within 48 hours all the new site cached, 90% of the pages indexed
The original 301 was done in php, last week i changed it to htaccess, no real big improvements but still down 70% of my original traffic, i did PM Everett through SEOmoz but i think he may have been too busy to answer as i wanted clarification of what exactly he did and what exactly he meant by “Swith to turn on the old Internal link URLs”
I am willing to try anything now as my traffic is terrible.
It seems to me that the home page is the problem, PR has returned to inner pages, but home page i show for my name, but you copy any text then nothing
Can you clarify what he means by turning on old link URL’s as it sounds like to me its turn off the 301 let the old site get crawled again then turn back on the 301.
I could be wrong but if thats the case ill give it a go, if he means just add a site html sitemap pointing at the old URL’s thats been there for a few weeks and no Joy
I had my Htaccess rewritten and its 100% 301
September 14th, 2012 on 7:02 pm
Paul
If you see the traffic loss is from your home page only can you isolate where the loss is coming from: What Keywords? What Traffic Source? Was their a drop in keywords?
You might have a rare case where the 301 redirect is not the problem and something else (possible penalty) happened at the same time.
Also double check the header response of the redirect with:
http://www.rexswain.com/httpview.html
September 14th, 2012 on 8:28 pm
Hi Steve
Thanks for the response, I did link the url but i think it was caught in your spam blocker
Its all keywords really from the 150 i was tracking previous only 4 are in the top 100,
we wrote our own header response checker and everything was perfect, I requested a reconsideration from Google and they confirmed they is no manual penalty.
The only thing i can think off is the original PHP redirect which we changed out 4 days ago for a Htaccess version so may have to give that a week or 2 to kick in if that is the problem.
I have catalogued every action i have taken to resolve this issue.
Today i added a blog post with 20 links pinging the old URLs to which i did see google go to 50% of those so fingers crossed .
site:search says there are around 500 pages from the 17k pages previous which is about right,
The thing i was worried about is if the php redirect was the problem 14 weeks of having 2 identical sites could be caught up in an algorithmic penalty and wont be resolved until the next penguin/panda update
Ill report back in 7 days for an update
September 14th, 2012 on 8:33 pm
Thanks for the update! Keep us posted.
Any chance you were building aggressive anchor text to these penalized keywords? Hopefully not cause if so then we have an ugly Penguin to mess with.
September 26th, 2012 on 8:50 pm
Quick Update
No real change in my rankings so today i decided to turn on the internal links, as mentioned In the SEOmoz post, i was going to do this last week but i wanted to make sure that i did not change too much too quick and not really know what really my issue if anything did at all.
3 hours after implementing i ran a quick URL update on 20 random keywords and 6 keywords i have not ranked for for months were showing, although pages 9 -10 it was a great improvement, it could be a coincidence so time will tell if they stick, Today is Wednesday 21st Sept, i am going to leave the internal links until Monday first of October and will report back.
The good thing is i have fully documented this whole 14 week process so the results will be quite interesting
Paul
September 26th, 2012 on 8:53 pm
Paul thanks for the updates and we are glad to hear the good news. Keep us posted cause I am curious to see if it was the 301 redirects or some other underlying issue I mentioned.
October 1st, 2012 on 3:47 pm
Hi Steve, beginning to think maybe you are right in regards to an underlying issue, after i have turned on the internal links, the bots went absolutely mad.
I am going to leave until after the next Linkscape update as i want to see a few things, plus 4 days is not long enough to see any changes, but i had my fingers crossed my analytics went like the grap in your post
And these are my worry’s
1: Google still sees my home page as a PR 1 the old page was a 4 and many of the internal pages PR has Passed over but not the home page
2: at first i thought it was Just Google, however SEomOz PA is 42 where the old page is still 60 (down slightly from 68)
3: I asked for a reconsideration request and Google confirmed it there is a problem they have no manual actions in place.
4. just looked at server logs and Google is visiting the old site
so in theory its not just Google slow at seeing results.
As mentioned i am going to leave another week then maybe consider reinstating the site which means removing the 301 which i really don’t want to do, but at least i had some traffic.
I have posted the URL in a forum to see if they can see anything and no one can
I get 301 where i want them and where they should be.
As of Friday when i updated the internal links all old pages were removed from Googles index so seems like a good thing to me
Is it possible if i pass you the URL you can have a quick Look and maybe fire me some questions and ill answer what i have done
Thanks again Paul
October 6th, 2012 on 12:17 am
Woot
Had a partial recovery today, that’s 8 days after turning on those internal links, it could be a coincidence im not sure, and i doubt it
after turning on those internal links bots went mad, however we have just had a panda update so that could have been the activity.
Early days just yet, and 40% of the keywords have recovered, not where they were originally but not too far off
Ill report back next week with another update and if the post lets me ill post some images as the peak of the bots is impressive
October 8th, 2012 on 2:55 pm
That is great to hear! With Google releasing a Panda, Penguin, and Exact Match Domain update all in one week it might be hard to correlate the recovery. Either way it is good to see progress being made. Please keep us posted.
March 8th, 2013 on 7:23 pm
Hi Steve,
I’m a fairly new developer and made a very rookie mistake: I forgot to apply the 301 redirects to the old, now-defunct site. Is there anything I can do? The old site has been removed from its old server, and I no longer have access to any of the files. However, Google is still indexing that site structure. I appreciate any help you can provide. Thanks.
March 8th, 2013 on 7:53 pm
Bill the best option would setup a new hosting account, point the old domain to new name servers and drop .htaccess file that contains the 301 redirects in the new server. This will reactivate the ability to at least do the redirects.
Now if you don’t have access to the old domain name servers then I would try to claim the old the domain in Google Webmaster Tools. There is a feature in there to let Google know that the site has moved to a new location.
Let us know how it goes!
April 26th, 2013 on 5:59 am
Ricardo,
I have been working on some marketing efforts for our eCommerce site. We don’t have anything that really lives in the http: world only https: world. I asked our server host to do a 301 redirect from http to https. Not sure if that was needed, but since then our site isn’t being indexed because it seems it is in a redirection loop.
I don’t know what to do. I have asked him to remove the redirect, but he said it is a little more complicated than that.
Thanks
April 26th, 2013 on 8:49 am
Hey Bart,
If there was a way for you to render your site in a non-secure way, or at least the pages that are more eligible for search, that would be best. However, the crawling / indexing process has evolved quite a bit and search engines, especially Google, should do fine with your https site. As long as the pages aren’t behind some virtual wall that would keep spiders from getting to, you should be fine.
My recommendation would be to undo your 301 redirect from https to http. What server platform is your site on? Linux? Windows? It shouldn’t be complicated to undo that, but it’s tough to tell without looking under the hood.
Anyway, I’d say if this is posing a bigger bite than you can chew, then don’t spend too much energy on it. Although it would be better to have those SEO friendly pages on a non-secure version of the site, at this point it would be more of a best practice thing to do. Google will probably be able to handle the https pages just fine.
Let us know how it goes!
April 26th, 2013 on 8:58 am
I appreciate the feedback. It all seemed fine until we did a 301 redirect of http to https. Since most of our site is product pages and associated with the cart we have the site in the https protocol.
We are on a windows platform IIS 7.
April 26th, 2013 on 9:17 am
No problem! Yea, I don’t blame you for trying, I would have done the same thing.. Now, Windows servers can be a bit more touchy with redirects as they have to be done through isapi rewrites and other more complicated methods rather than a simple line on a .htaccess file in a Linux platform. That explains why your server host is saying it’s not that simple to undo it, but that doesn’t mean it’s not doable. Try to get this undone and move on from this issue. Sheesh!
Good luck, and keep in touch.