How to Best Track Data Using Search Console

It’s important to track data in Search Console, but what should you look for, and how often should you look at it?

Video Transcript

David: Okay, so Dave, you asked another question. Do you recall the second question?

Dave: I do because I think I just did it yesterday.

David: Okay.

Dave: Alright. So, in the Search Console, when you’re looking at the pages, there are the different categories that Google says about your pages, such as not indexed, redirect, and all that kind of stuff, right? And you’re looking at it every week, and some of the stuff is perfect. Like, I don’t want my account pages indexed, or I don’t…

David: Yeah, yeah.

Dave: But I have to click on it and drill down every time to look at that. Is it best to just export everything to an Excel sheet and then export it next week and do some kind of comparison between them to see what’s different?

David: So, weekly data exports from Search Console would be too frequent for most sites because the data doesn’t change that dramatically from week to week. So, what I’ll do is log in and look at Search Console once a week and decide if there is an action I need to take. If there is an action I need to take, let’s say there’s a sudden influx of 404 errors. I will add that from my queue, do it, and then hit the validate button in the Search Console. That tells Google, hey, I fixed some stuff. Will you go back and look? That can take weeks. So, there’s no reason to pull the data every week if Google hasn’t validated my change, right?

Dave: Right.

David: So, sometimes I’ve seen it take a month before it gets back, and inevitably, it gets back and says it didn’t pass, which is not a great message that they deliver to my clients either. Right? It’s like, thanks, Google. I just cleaned up all these 404s, and you’re going to deliver a message to the owner of the Google Search Console account that whatever I did didn’t pass? In reality, it did most of what I wanted it to do, but it’s never going to be perfect. So, I don’t appreciate that, Google. That’s just not cool, man. But, so for the week-to-week catch-ups, even though I recommend you look at them once a week, I’m not saying you fix all those problems once a week. But when you do identify a problem that’s a priority to be fixed, use the validate button, and then don’t pull that data until Google tells you whether they have completed their validation or not.

Dave: Okay.

David: So, let’s say you clean up a bunch of 404 errors, you hit validate, and eventually, you’ll get a notification saying, hey, the validation is complete. Then it’s worth looking at that and seeing the difference.

Dave: Okay.

David: But if there’s not a problem like you said, there are certain pages you know you don’t want to be indexed, right? Then what you’re really looking for week to week is sudden increases or sudden decreases, right? So, you’re not really looking to account for every single page on the site every week. You’re using Search Console to see if something significant happened.

Dave: Okay.

David: Right? So, I have a client, and someone is constantly pinging their search query bars with Chinese characters. And so, because I’ve set up, and WordPress does it by default, set up searches to be no indexed, my no-index pages are growing and growing and growing and growing. So, I went to the developers and said, hey, weren’t you going to be installing a security plugin on this? They’re like, oh yeah, yeah, I forgot, sorry, here it is. Right? Okay, great. So, now I know that the reason they’re pinging it is because they want to hack the site, right? They’re trying to do insertion. Well, now, even though the numbers keep growing, I know why. I can explain it. It’s less than ideal, but it’s also not really hurting the client because, presumably, the security plugin is protecting them from a brute-force attack. Now, I will eventually return to that and go, alright, something is going on. Maybe we need to block IP addresses or something like that to figure it out so they stop. However, the process I followed was to look for growth or declines that I needed to address. Not every page that I want to be indexed is still indexed.

Dave: So, then, when you’re looking for growth or declines, are you taking a screenshot of Search Console and comparing it week to week?

David: No, because with Search Console, the data goes back several weeks, even several months, so you can see trends.

Dave: So, you’re looking at the graphs?

David: Right.

Dave: Okay. And because I’m looking at the graphs once a week, I can see, well, okay, for the last seven days, it’s been pretty flat. Or last Wednesday something spiked or declined significantly. That’s really what I’m looking at: trends. If I notice something has changed at that point, I’m going to deep dive and pull it. But I’d be hesitant to pull that data and archive it because it’s never going to be comprehensive and complete. It’s going to be what Google decides to share. And Google doesn’t always index every page. Google doesn’t always provide all the data. So, if you were to go and be like, oh my, this page is gone, well, that doesn’t mean it’s really gone. It just means it’s not in the data. You know my opinion about paying for tools, and I’m trying not to sell additional tools. But if you have money burning a hole in your pocket and you’re a developer, Screaming Frog is worth every dime you pay for it. And it’s not in the Curious Ants process, but you all get the secret, secret insights. I run Curious Ants once a month on every client because that’s going to give me a real-time, up-to-date, accurate statement of how many 404 pages I am internally linking to, where they are, and what the anchor text is. And then I can literally export that, and one day or the afternoon, I’ll put on nineties alt-rock and work through all hundred internal links to 404 errors and just fix them all. But that, to me, is a better tool for going through the entire site and seeing all the potential problems. And when I fix the 404s, or I fix the redirect loops, or I check to make sure the pages that are not indexed are actually the ones that I don’t want indexed in Screaming Frog, and I fix them or whatever, then I see the data a couple of weeks later in Search Console because those problems were fixed.

Dave: Okay.

David: Right. So, for what you’re doing, the better tool is probably not Search Console. The answer is probably Screaming Frog.


Have a question about this process? Ask it here:

Get started doing SEO today

SEO seems hard- you have to keep up with all the changes and weed through contradictory advice. This is frustrating and overwhelming. Curious Ants will teach you SEO while bringing your website more traffic and customers- because you’ll learn SEO while doing it.