Reverse engineering in SEO is the art of learning what others are doing to rank their content on Page #1 of Google.
We live in a world striving to be original. Copyright and Intellectual Property regulations are set in place to protect creative work and authorship. It's made us cautious about what we do. And for good reason.
But reverse engineering is not copying. In fact, it's what a child does when taking apart his favourite toy. It’s learning how something works.
Learning cannot occur without reverse engineering. In any industry, it’s how we create new and better products. Studying and figuring out how a website has been structured for SEO or what content marketing strategy has been used is a powerful way to get your content on Page #1 of the search results.
Let’s look at how Wikipedia describes reverse engineering -
“Reverse engineering, also called back engineering, is the process by which a man-made object is deconstructed to reveal its designs, architecture, or to extract knowledge from the object" - Wikipedia
The keyword here is “knowledge”.
Knowledge and ideas are not protected by copyright or trademark laws. And that includes your competitors’ SEO strategy.
Reverse Engineering in SEO
I and my team use reverse engineering to speed up the SEO process for our clients.
There’s no need to reinvent the wheel and certainly no time to repeat someone else’s mistakes. What’s working and what’s not working has already been discovered. And the answer lies amongst those websites that are already ranking well for your topic or keyword. All we have to do is replicate what works and combine it with our own battle-tested SEO strategies.
Tools That You Need
1. Ahrefs / SEMrush
For the purpose of this tutorial, I will use Ahrefs. If you are a fan of SEMrush, feel free to use it. Both tools do the same thing and they will cost you about the same money.
2. Screaming Frog
Screaming Frog is a crawling software that will let you see a website through the eyes of a crawling spider. They have a free version which is quite adequate. But if you are working with large websites, I highly recommend getting the paid version.
Start By Choosing the Winners
The whole idea of reverse engineering is to replicate what already works best for someone else. Choosing competitors that are doing great in organic search is a crucial step for achieving that goal.
The process I outline in this article can be used whether you’re an agency working for a client or a blogger working on your own website.
If you are working on a client’s website, it’s worth asking the client to provide you with a list of their direct competitors. You can also do your own research by just using Google (and I highly recommend it).
I suggest starting with a bigger initial list. Work with at least 5-6 competitors. For better results, you can extend the number to 10-12, as during the process some of them will fall off the list.
Next, we are going to filter those competitors and shortlist the ones that are doing well and are direct competitors.
Open your Ahrefs, run each competitor through Site Explorer, and look at the Overview data. At this stage, we only want to pay attention to the number of organic keywords and organic traffic.
You should consider the industry and the demand when you assess how well your competitors are doing. The above screenshot represents a local law firm with a single location.
Looking at the numbers above, I may consider them being worth reverse engineering. The only thing I would like to check is why their organic traffic had dropped from 2K to 1K in June 2019.
If I go to their backlink profile, I see that at the same time the traffic dropped, their backlinks have spiked, so most probably there are some bad links built at that time, as the correlation is obvious. I will note that, so I know to be careful with their backlinks.
Let’s do the same exercise with another website:
The above competitor is giving you even more data to work with.
When you shortlist your competitors, you also need to be mindful if they are direct competitors of yours. The above screenshot, for instance, represents a legal firm with a national presence and locations in four major cities. Some of the keywords may not apply to you if you only service one of those cities.
Every case will be different, so be mindful and open to further explore the information, if needed. You can still use the competitors that have extended offers if a decent percentage of their organic traffic comes from keywords related to the services or locations that overlap with yours.
Repeat the above process for all your competitors and make a note on the ones that perform the best organically and are directly competing with you.
Speed Up Your Keyword Research by Stealing Your Competitors’ Keywords
Once you have created your list of winners, you are ready to dive into keywords and rankings.
From the Overview page in Ahrefs, click on Organic Keywords located on the left-hand side of the screen. Then export all the keywords. Repeat this for all of the competitors in your winning list.
Once you are done exporting the keywords, open each file, and combine the data from all your competitors into one master keyword sheet. Don’t worry about duplicates, we are going to clean up that in a second.
I usually get rid of some columns that the default export contains, so my master keyword sheet looks like the one below.
Of course, it’s totally up to you if you want to keep the default export as it is.
I have deleted some columns, but I also have added a new one – Search intent.
At this point, your list may consist of 10,000 keywords or even more, so we will need to clean up the duplicates. If using Excel, you can follow this tutorial to remove your duplicate keywords (make sure you only clean up the selected column) or if you are using Google Sheets, check this one - remove duplicate values in Google Sheets.
Now it's time to fill out the Search intent column.
I use three options here:
Searches with commercial intent are the ones where users are ready to purchase or buy the product or the services they are looking for. Most of the time, the CPC column will help you determine commercial intent – usually, the higher the value is, the higher are the chances that a click becomes a transaction.
Navigational keywords are usually branded searches, and Informational keywords are keywords people use to research a certain topic.
Ideally, you want to target as many commercial keywords as possible.
Now, you have your master keyword list which includes all the metrics that will influence your decision on whether you want to go after a certain keyword or not.
Explore Competitors Top Pages To Get Ideas on Hot Topics
You can even go further and explore what topics bring the most traffic to your competitor’s website.
In Ahrefs, click on the button, which is located under the Organic Search menu on the right-hand side of your screen.
You can sort the pages by value or traffic to find a good topic to cover on your website.
Find Backlink Opportunities
Again, I use Ahrefs to find backlink opportunities.
This time, click on ‘Backlinks’ on the left-hand side of the screen. Then filter the results by only One link per domain and 'dofollow' links.
Export the file:
I will now clean up the sheet, leaving only the column that I need. My working sheet usually looks like this:
As you can see, I have deleted some of the columns in the export file leaving only the data that would help me determine whether I would like to build that link or not. I have also added a new column “Opportunity” which I will fill out as I go through the list and further explore each link.
Blogs and news websites, where content creation would be required, I consider content marketing opportunity, while citations, forums, or other websites, I mark as a link building opportunity.
Of course, going through each link is time-consuming, but hey… SEO is not a one-day job - your competitors didn’t build their authority in a week. The sooner you start, the better.
Reverse Engineer Your Competitor’s Site Structure
Site structure shouldn’t be neglected, as a logical and hierarchical structure will make a website easy to navigate by both search engines and users.
The smart tactic when it comes to site structure is to build it for growth, but some sites have not been built that way.. So, if you need to restructure a website, there is no need to start from scratch, reinvent the wheel or try to be original. You can use what is already working.
The best way to find how your competitors have organised their website is to use Screaming Frog visualization tools. My favourite tool is the Visualisation Path as it gives you a hierarchical view of all existing folders and subfolders.
I like to combine the directory tree visualisation with the directory tree graph to help me visualise it better.
Reverse engineering your SEO can bring enormous value and get your pages ranking much faster.
It shows you exactly what keywords your competitors are targeting, what topic and keywords bring the most traffic to their website, how they organise their content, and how they have built their authority.
This is valuable knowledge that helps you know you are on the right track. Reverse engineering your SEO is a clever strategy that can save you months or years of trial and error.
This post was most recently updated on September 3rd, 2020