Bulk Check How Many Competitors Are Targeting Your Keyword
In December, we made public our inhouse Keyword Difficulty Checker.
It sounds fancy but essentially it is an Excel spreadsheet with our proprietary formula. We have done some SEO experiments and it does a pretty good job predicting the keyword difficulty.
One of data required for the Keyword Difficulty Checker is the “intitle:” result from Google and collecting this data manually is a pretty cumbersome task. This is why I want to share with you few alternative methods to bulk collect this data.
Before I get into how to bulk collect the “intitle” data, let me explain what it is.
Page title can tell us who are targeting the same keyword
By using Google search operator “intitle”, we can see how many other websites contain the keyword you are targeting.
For example, when I demonstrated how to rank for the target keyword “keyword difficulty checker” without building backlinks, one of the things I did was to use “intitle” to make sure there are not too many other websites (= competitors) going after the same keyword.
The logic behind this is based on the data published by Searchmetrics. According to the data, there is a positive correlation between the presence of target keyword in the page title and the ranking position.
In other words, when a page has the target keyword in its page title, it is more likely to rank high compare to a page that does not have the keyword in its page title.
This is why by knowing the “intitle” number, we can see how many competitors we have to go against for the keyword we want to target.
How to bulk check “intitle” data?
Although it might sound smart that we incorporated the “intitle” data in our Keyword Difficulty Checker, this practice has been done for quite sometime in the SEO community.
This is why there are a couple of bulk checking tools (as far as I know) that have been available in the market for quite sometime.
Lovepage1
I have not personally used this software but based on the videos and descriptions on its website, it seems to be a user friendly option than the one I am going to show you in my video.
ScrapeBox
This is a must have tool for anyone who is serious about SEO. Many people say it is for Black Hat SEO but this is not true. The reason it is associated with Black Hat SEO is that many of its functions have widely been used in Back Hat SEO techniques.
In my opinion, it’s a tool like knives, when used correctly it could be very useful.
Note on operating cost
In order to run these software efficiently, you will need to get private proxy. Otherwise your IP address will get banned by Google for good few hours every time you run the software.
Here are the two websites I have got proxies from.
BuyProxies.org
When I only need a small number of proxies, I would just head over to buyproxies.org.
I personally have only used their dedicated proxies and they are pretty good. There were a few times that I had to contact them to get me a brand new set of proxies because the ones I had all stopped working. They responded quickly without any problem getting the replacement.
http://www.ipfreelyproxies.net/
For a larger project, I would use ipfreelyproxies.net since they have minimum order of 20 proxies. One good thing about their service is they don’t send you a new set of proxies every month unless there is a problem.
Also, they use IP verification so you don’t need to worry about your proxied getting abused by other people. Since I work with other team members and showcase proxies on video, I personally like this feature.
How many proxies do you need?
The cost of private proxies varies as you can see from the two websites and it really depends on how many keywords you want to check in one go.
As I explained in the video, ideally you should use each proxy at least 20 seconds apart (personally, I recommend 60 seconds to be on the safeside). I have explained how to determine the required pause time in the video.
You can also calculate the time required to complete a task by doing a simple calculation.
This will give you a better idea how many proxies you should get.
Let’s say we have 100 keywords to check and have 5 proxies. We need to pause 20 seconds in between so the calculation is
100 keywords x 20 seconds / 5 proxies = 400 seconds = 6 minutes and 40 seconds.
If we have 20 proxies available, this becomes to
100 keywords x 20 seconds / 20 proxies = 100 seconds = 1 minutes and 40 seconds.
Video: how to use Scrapebox to bulk check “intitle” data?
I build websites and use them for testing various SEO strategies and tactics.
LinkedIn Profile
- How To Track Keyword Ranking? - April 27, 2020
- Is Updating Old Content For SEO Worth It? - March 30, 2020
- What Is Remote Desktop Access (a.k.a RDP Or VPS) For SEO? - March 27, 2020
Thank you! It’s been years since you wrote this post but I found it in 2019 and it is still valuable for me. I’m going to buy ScrapeBox and a few proxies and start getting those allintitle: values in bulk. Also, using a remote desktop solution to provide access to my tools is a genius idea! Some tools will block your account if you access them from multiple locations but a remote desktop solution will give easy access to anyone anywhere anytime!
Hi Tony,
I very much enjoyed reading this article. There is a Bulk Allintitle Checker tool (https://allintitle.co) that we use. I thought it would be a good idea to share it here. It can help people perform allintitle research for a long list of keywords in under a minutes without the need to manually search for every keyword on Google. Hope it helps someone here.
Best,
Hi, the video is down :(, it’s been a year. I’m with ScrapeBox, proxies loaded in front of me and ready to roll! I’m also interested in creating a VDI (remote desktop). I found an excellent server solution and I also have a Windows Server license ready. Is there any way for me to access these videos again? They are reaaaaly good! Thank you!
Sorry about that David. The video is up again now.