Data Extraction Errors
0. Understanding Error Messages and Screenshots
A ConnectionError message appears when accessing a page fails. This can occur if the server you visited is down or too slow. It can also happen if the Listly server's IP address is blocked and the website you're trying to extract data from returns an error.
A NoMatchingtabError appears when tab matching fails or, similarly, when page access fails. In many cases this is due to the Listly server's IP address being blocked and the site returning an error.
PartSelectionError is another error message that can occur when the Listly IP address is blocked and the specified part cannot be extracted.
First, don't panic. Check the error message and then review the screenshot.
If the screenshot image looks like the example below, there is a high probability that the IP address has been blocked.
1. Slow down the execution speed as much as possible
There are popular websites that attract users from around the world. These sites receive a lot of abusive traffic from across the globe. When collecting data with Listly, if the collection speed is too fast, the site may interpret the attempt as abuse and the data extraction will fail. In the worst case, the user's IP address could be blocked. To prevent this, the Listly team has added a blue bar at the top right of the dashboard so users can adjust the execution speed.
2. Set up a proxy server
A proxy server is a server you route through in the middle when accessing a site. The reason for setting up a proxy server is to hide the visitor's IP address.
Why hide the address? Because sites you access can impose various restrictions based on IP addresses. The following situations are typical.
When a site is restricted to access only from certain countries
When the number of accesses to a site is limited within a certain time period
When specific IP addresses with bad records are restricted
Listly Premium Proxies are proxy servers Listly purchased from overseas providers. They are shared servers used by all users.
Currently we provide servers in the US/Germany/Netherlands, and the countries may change over time.
2-1 When blocked even with a shared proxy: set up a dedicated proxy server
If you used a shared proxy but still see a Connection Error/Empty Data Error message and a screenshot indicating you've been blocked, what then?
Even if you set up a shared proxy server, you have still been blocked.
The default proxy servers are shared proxies used not only by you but also by many other paid users.
Even if you did not use a particular shared proxy, another user could have used that proxy server to extract the same website and gotten blocked, causing issues for you.
Therefore, to eliminate the risk of sharing a proxy server with others, it is better to use a dedicated proxy server.
A dedicated proxy can be used exclusively by the single user who requests it, allowing stable server configuration without worry of blocks caused by other users.
Also, if you need a proxy from a country other than the ones Listly provides by default, it is better to use a dedicated proxy (e.g., Eastern Europe/Middle East/Latin America/Oceania countries).
Of course, even with a dedicated proxy, if you continuously collect very large amounts of data very quickly, there is still a chance of being blocked.
In that case, we can switch you to a third-party proxy once at no additional cost or purchase and set up another proxy server for you.
At that time, you can subscribe to a dedicated proxy server solely for you.
Please email [email protected] with the desired country's proxy server and the estimated monthly data extraction volume.
Example) Japan proxy server / expected 7,000 extractions per month
Example) South Korea proxy server / expected 500 extractions per month
Monthly subscription fees start from 60,000 KRW.
However, while higher-quality proxy servers are less likely to be blocked, their price increases proportionally with quality.
3. Reduce the number of URLs for group extraction
Assume you want to perform a group extraction by inputting 1,000 URLs.
But if the platform you are accessing already defends against numerous access and data extraction attempts, the more URLs you include in a group extraction, the more it may be considered an attack.
Therefore, when adding URLs for group extraction, rather than extracting them all at once, we recommend splitting the URLs into smaller batches and extracting them gradually.
For example, if you tried group extraction with 1,000 URLs and it failed?
We recommend splitting them into 100 or 200 and performing multiple group extractions to prevent being blocked.
Last updated
Was this helpful?