Learn How to Find 404 Pages and Fix Them
Finding and fixing 404 pages on your website or your clients is necessary, and here is how to do it.
Video transcript:
David: So, Tricia, you asked about the Weekly Google Analytics dashboard. Okay. So, this is a screenshot you shared from a snippet from that report. The whole idea of this weekly Google Analytics report is to instantly help us identify if something has gone wrong. It’s not really to judge success as much as identify a potential problem. And so one of the things we want to check in this is whether there is a sudden increase in pages that were not found. And apparently, in your report for one of your clients, there is a 160% increase in pages not found. Okay.
Tricia: That’s a big increase for me based on what I’ve been looking at on their website. I do think that they had someone redo some things on their website, so I’m wondering if they just didn’t get a page forwarded properly.
David: Right. Okay good. All right. So, I want to make a couple of caveats about this. So, number one, hooray! Yay! You discovered it before it became a real problem. That’s one of the purposes of this weekly report. It’s why I start every Monday morning by looking through these for all my clients. Two, we have to remember what this means. This is a Google Analytics visit to a page that’s not found. So, it means that 26 visitors or users have visited at least one page that wasn’t found at least once. It could mean that there were 13 pages not found that were each visited twice, but because this is visitor numbers, we don’t know how many pages. But we are using this as an alert to tell us that we need to look further to investigate this because people are coming to pages that don’t exist.
Dave: So, these are not really pages not found.
David: Right. Yeah, you’re right. This is misnamed. You’re right. This is users visiting pages that were not found. So, this dashboard is designed to create an alert that we can then take action on. But it does not necessarily give us enough information to know where those pages are, right? Frankly, I’d suggest that Google Analytics is probably one of the less helpful places to look for 404 pages because the reason we want to clean up our 404 pages has to do with Google, not just users.
Tricia: Okay, so I was thinking of Google Search Console…
David: That is exactly where I would go.
Tricia: Okay, good. That’s what I thought. But then I was like, well, this is a Google Analytics report.
David: Yeah. This is trying to warn us that pages have not been found in a weekly repeatable report. The whole point of this is doing exactly what you’re doing. Where are those pages? And what do I need to find? Right? So, the first place I would look is Google Search Console under the indexing report. And then as you get there, you will see beneath, you just scroll down, it says, why pages aren’t indexed, and you should see something. There are several options. One of them is not found 404. That’s what this is probably. We also have to acknowledge that Search Console does not have as immediate data as Google Analytics. So, we might not see in Search Console yet any 404 pages, but we should at least start by fixing the 404 pages we find in Search Console. And the fix for this is to set up a 301 redirect from the old URL to the new URL. Now, there are some tricks with doing this because you’ll need to really evaluate if a page is supposed to exist because sometimes you want a page to 404. Right? For instance, a lot of times, on my sites or my clients’ sites, like an archival page, category page 40, will 404. Well, that’s because there are no longer 40 pages in that category anymore. And so, page 40 truly shouldn’t exist. So, I don’t want to redirect page 40 because, eventually, there will be 40 pages of posts, and I would like Google to read the 40th page.
Tricia: Okay.
David: That’s one of the things I do is I purge those out. I would also pay attention to things like query parameters in the URLs. Typically, those are harder to redirect. But if you do see an important page, then I would set up a 301 redirect from that old URL to a new and the most relevant page on your site related to that. There should be something that’s close, even if it’s just the blog page. But if nothing else, then go to the home page. But don’t just redirect everything to the homepage.
Tricia: Okay. I’ve looked at the not found 404. But one question is, when I look at why pages aren’t indexed, I have not found 404 is one of the things, and then soft 404. So, when I look at the 404, I do find one of them looks like an employee. So maybe they’re not an employee, or maybe they need to redirect. So, I’ve got kind of a list of these that I can check. Good heavens, there are like 100 of them.
Dave: Right. So, I think that’s a really good example, Tricia. So, if an employee leaves a company, what do you redirect to? Yeah, you really don’t need to, probably.
David: I would.
Tricia: I would redirect to just the employee page. And I don’t know, I have to look at this, but they should have an employee page.
Dave: Depending upon what you’re thinking about, that’s one of the reasons why it’s important to have a good 404 page on your site. So, you’ve got some main quick links that they can go to. Because if they’re looking for a particular employee and you redirect them to the employee page. Maybe that’s what you want to do. Maybe not. I don’t know.
David: We have to remember why we’re doing redirects. Yeah, it’s not just for users. It’s also for Google. So, one of the most important things for SEO is link-building. And we haven’t talked a lot about that, but it’s super important. If there is a link from another website to your website, then the page that no longer exists before a 404 page will no longer give you the value of that link.
Dave: So, Google will see that, and…
David: They won’t count that link anymore for the benefit of your site. That’s why you need to 301 them.
Dave: So, that’s one of the reasons why in link-building, if you’re linking to an ecommerce site, you don’t want somebody else to link to a particular product, for example.
David: Well, you just need to handle the product to make sure that it fails gracefully.
Tricia: Right. Yeah. You need to redirect that product from not where there’s no page to something to say this product is not available; here are some options.
David: Right. Like a category page. A good ecommerce solution will not 404 if it’s out of stock, but I forget it offhand, but there is a proper header code for when something is temporarily not available. Or, what I would do if it’s temporarily out of stock is I would make sure the page still returns to 200. So that later people can still find it, and it doesn’t have to be reindexed when you now have stock.
Dave: So, what if you have a client that says I’m not selling this product anymore, and I don’t know if I ever will? But they don’t want it to appear in a directory and all that kind of stuff?
David: 301 it. Kill the page and 301 it.
Dave: Would you kill the page, or would you just place the product in draft mode?
David: It’s functionally the same from a search perspective. But yeah, the primary reason is Google. If this is an employee page, and they’ve been profiled on The New York Times, and there’s a link to your website from The New York Times, if it 404s, you don’t get credit for that anymore.
Tricia: So, here’s the interesting thing. I’m just looking here on the client site, and the 404 error goes to the website.com/employees and then the name. When I go to the actual site to look up their team, they’ve changed it from employees to AboutUs/OurTeam. Because I’m looking and there are all these employees. So, I think what they did looks like they did not get that employee’s page forwarded. And I don’t know if I mentioned it or not, but this is a gym-type facility, so these are trainers. So, these trainers might actually be written up locally as doing something. And so, they may have a link. So, they are losing that. So, yeah, I’m going to do a little more checking and see, but there are a few other bad links, but that’s the majority of them. So, I can see that this is like three-fourths of what happened to the employee, then there’s also…
David: Let me interject a second here with two things. Number one, you can do a wild card with the redirect. Let’s say they changed the directory structure and moved employees to about/employees. You could do a wild card redirect to say hey, if anybody asked for employees, send it to about/employees. Now, warning, you have to write it correctly because you could create a loop if you don’t write it properly.
Tricia: Yeah. And the one thing, too, that I notice is that it looks like it’s harder to find that way because they’ve got different locations. Okay. I’m just looking at their page here now to see if they do have an employee page… I wonder if maybe they… I don’t know. I’ll have to look it up and see. But they’ve got locations. So, if I just send it to the employee page, they would still have to do a lot of searching. They can’t just look down and say, oh, here they are. So, I think the wild card for this specific one might not work as well as the one that had them all listed.
David: Yeah, but this brings up another very important point. We really don’t want to change URLs unless we really have to because of this kind of mess we can create. We have to keep redirects in place for at least a year, if not longer. I usually keep redirects in permanently. That’s just to protect myself and my clients. So, if there are no backlinks to a URL, it will clear out of Google’s system in a year, but if there are backlinks… So, just don’t change URLs is a better practice unless you really have to for some legal compliance reason. And when I say don’t change URLs, I mean not even a single character. If you change from employee to employees, plural, that is a change, and that’s going to cause you a headache. It’s not wrong; it’s just a headache. I’ve worked really hard with developers to redesign websites and keep the URL the same to avoid the nightmare of having hundreds, if not thousands, of redirects on the website.
Tricia: Yeah.
Dave: Here’s a question on that, though, because we had a site that was done in Microsoft servers, something, whatever… They were search results .aspx or something like that. And obviously, WordPress doesn’t support that. So, I remember asking some questions. I think the best that we could do was to do some redirects for some of those types of pages. So, I guess the question is if Google has the .aspx do they care about that part of the URL?
David: Yes.
Dave: Okay. So, we have no choice?
David: Yeah, you have to do the redirect. So oftentimes, this is ASP, the language ASP as opposed to a php-built website. This is an ASP-built site. When I started in SEO, the company I worked for built websites in ASP. Some of these legacy ones can actually be more valuable to redirect because those links have existed for ages. Now with WordPress, I use a plugin called Simple 301 Redirects. It’s a great plugin with low overhead. I don’t want a plugin that also tracks 404 errors because that creates bloat. If it was only up to me, and I had access to the server, I’d rather put all the redirects in the htaccess or nginx file. That doesn’t depend on WordPress, so you can usually end up redirecting more file types, for instance. It’s really hard to redirect query codes using some of these plugins, but sometimes you need to. If you have access to the server and can add them there, that’s a better place. Plus, it’s a lot less likely that somebody removes it, or there’d be a glitch with the plugin or something like that. But simple 301 redirects if you’re using WordPress. The problem is like even last week or this week, I was working with one of my client’s sites, and I ended up adding another 50 redirects just because the site’s so old and hadn’t been done in a long time. Now, this simple 301 redirects page is full of redirects. It’s cumbersome but important.