google search 18th of May, 2020

Status
Not open for further replies.

Petr

Administrator
Staff member
Hello,

We have just issued an update regarding ReCaptcha solution on a google.com/sorry/index webpage.
https://2captcha.com/blog/google-search-recaptcha

Unfortunately it didn't work very well. We do provide some valid and working tokens now and google search parsing started to work again, but we still have some problems with tokens too. We are investigating the issue further. Our goal is to reach our usual 98%-99,5% amount of valid tokens.
We will keep you informed about latest updates regarding this issue in this thread.
 

Petr

Administrator
Staff member
Worker's cookies
First thing we did was useless.
One of our competitors insists that you've got to set a worker's cookie which worker receives when solving a captcha. Because of that lots of customers demand this functionality. Ok, we did it.

Now, when requesting a captcha in JSON (add &json=1 to your request), we will return all worker's cookies he has when a token was made along with captcha resolution.

1590760840063.png

What we do now.
Ok if we started work around cookies, now we also support sending customer's cookie to worker. So when worker solves a captcha he has the same cookies as customer who requested a solution.
 

Petr

Administrator
Staff member
UserAgent
You may send us a UserAgent parameter. If you do, a worker will solve your captcha with your UserAgent.

Use UsarAgent parameter to send it.

Proxy
You may also send your proxy. It will be set on a worker's software before he loads your captcha so all requests could be performed via given proxy server.

According to reports, using of a proxy could make solving rate up to 100%, but after 10-20 captcha solved via given proxy new tokens became invalid.

How to make a request to send a proxy
Add "proxy" parameter in following format:
Code:
login:password@123.123.123.123:3128
and "proxytype" parameter indicating type of your proxy: HTTP, HTTPS, SOCKS4, SOCKS5.
If your proxy has access control, add our IP address 138.201.188.166 as allowed.

It is safe to send us your proxy. We will never use it and our workers will not be able to detect it, as we only use it on our gate 138.201.188.166 and we provide a temporary login and password to workers.
 
Last edited:

Petr

Administrator
Staff member
Receiving your cookies in in.php
If you want our workers to send your captcha requests with your cookies, you may send it to us as follows:

Add a "cookies" parameter to your request to in.php. ":" separates cookie name from it's body and ";" separates different cookies like this:

Code:
cookies="ANID:AHWqTUkiE1lX;NID:204=SbYHJRGMb4wtUG2"
 

Petr

Administrator
Staff member
What now
1. ERROR_CAPTCHA_UNSOLVABLE
Unfortunately we close a lot of captcha with this error. This is because a captcha could not be loaded in worker's software. And we can't load it again and give to another worker because token will not be valid then.
This is our concern for now. Note that we do not charge you for that, the funds for such requests will be refunded.

2. Amount of incorrect tokens
It is not that high as before but we are still working on improvement.
 

Petr

Administrator
Staff member
Send your cookies and proxy
Suddenly recaptcha solving when using your cookie and proxy resulted in 100% valid tokens! Please read of how to send it in our posts above.

Important!
1. Do not use cookies you received from 2catpcha with a token.
Send us cookies you've got on your browser for google.com host along with your request to in.php.

2. If you don't have cookies from Google.com, for example, if you parse from host google.sm, then you parser never visited google.com and you don't have it's cookies. Then you need to change an algorithm. First go to google.com and get a cookie once per session. Then when you've got to bypass captcha, send these cookies to us along with captcha solution request.
 
Last edited:

Petr

Administrator
Staff member
How you get google.com cookies if you parse google.sm and has no google.com cookies?
How could you get cookies from google.com if you are solving on another website and parser never goes to google.com before captcha arrives?
For example, you are parsing www.google.sm and you don't have google.com cookies. All you need is to open google.com and save it's cookies. Then, when you parse on google.sm and receives a captcha, send it to us.

Important
If a token doesn't work for you or if we didn't solve a captcha for you, you can't just try to solve it again.
Instead, you must go back and get a new captcha from search. If you don't do that and will try to send same captcha again, your IP address will be banned by Google.
 

Mark Miller

2Captcha Engineer
Staff member
FAQ on the topic:
1. Can recapthca be bypassed on google search?
Yes.
2. Do I have update my code/software?
Yes.
3. What exactly should I do?
Our suggestions:
  • If you use browser simulation - you MUST block execution of javascript from https://www.gstatic.com/recaptcha/ and https://www.google.com/recaptcha/api.js, or just load only HTML code of the page outside the browser
  • you MUST provide data-s parameter
  • it is RECOMMENDED to provide your proxies and use them to interact with google.com, rotate proxies after receiving 429 response
  • it is RECOMMENDED to provide your cookies to our API, we will use them on workers' side to solve the captcha. If you don't have cookies set by google, then open google.com to get cookies before solving a captcha.
  • it is IMPORTANT to build a proper final URL, that should look like: https://www.google.com/sorry/index?q=Q_PARAMETER_VALUE&continue=CONTINUE_VALUE&g-recaptcha-response=TOKEN then open the URL using the token and cookies
 

Petr

Administrator
Staff member
Cookie
We conclude that cookies doesn't matter, again. Moreover, part of our workers' software doesn't send cookie back, that's why sometimes we do not send it to you if you requested it.
Now our tests shows that cookie doesn't matter. It makes no better if you use it.

ERROR_CAPTCHA_UNSOLVABLE
We are working hard on lowering unsolvable cases along with lowering the time the error returned. But this error is still bothering us. We are still investigating a ways to eliminate it. By the way, using a proxy sometimes makes it worse because if the proxy you have sent to us is not good enough and worker lost a connection during a resolution then your captcha will not be solved. Also a worker would not gain a reward for the job too, which is rather disappointing.

Is there a hope?
Absolutely yes. Now we are working on a solution which will guarantee you a working solution. We need few more days to develop it though. Stay tuned and you'll be amongst first to know about it.
 

Petr

Administrator
Staff member
What news to the end of the week
Cookies - doesn't really matter.
Proxy - doesn't really matter.
UserAgent - doesn't really matter.
data-s - matters.

Current situation
There is still a problem with ERROR_CAPTCHA_UNSOLVABLE. We close a lot of requests with this error. And it means you've got to refresh a page and send us a new data-s to solve a captcha. Hopefully we will find a workaround next week.

Percentage of valid tokens is back to 95%! Finally!
 

Petr

Administrator
Staff member
New method to bypass a captcha on google.com/youtube.com/google maps and other google services

You are sick and tired of ERROR_CAPTCHA_UNSOLVABLE just as much as we do. So we keep searching.
Today we want to show you two new ways to bypass a captcha challenge on google.com.

Summary
Method 1. A new parameter "test=v"
Data-s is NOT needed anymore, proxy is a must.
In response you'll get URL containing GOOGLE_ABUSE_EXEMPTION. Open it to continue.

Method 2. A new parameter "test=r"
Data-s is NOT needed anymore, proxy is a must.
In response you'll get Q and g-recaptcha-responce. You know what to do with it.

Details
How anti robot protection works on google.com.

You are already aware of that but let us repeat the theory again.
0. First, when you try to open a page with search results (for example https://www.google.com/search?q=kurwa) but search engine decides that you are a robot and
1. redirects you on a page https://www.google.com/sorry/index?...ogle.com/search?q=kurwa&hl=en&q=yet_another_key
2. you have to solve a captcha. After bypassing captcha challenge you have to send a POST-request to https://www.google.com/sorry/index containing parameters:
g-recaptcha-responce: recaptcha token
q: yet another key from URL
continue: address where you got to a challenge at and where you are returned after completing the challenge, in our case https://www.google.com/search?q=kurwa
3. If the recaptcha token is valid then you will be forwarded with 302 redirect to a URL like that:
https://www.google.com/search?q=kurwa?google_abuse=GOOGLE_ABUSE_EXEMPTION=ID=db6d3789363scd78:TM=2567261522:C=r:IP=
74.125.228.111-:S%3DAPGng0s1iSgCZ-UiorwtuR7IPXL2bgRreA%3B+path%3D/%3B+domain%3Dgoogle.com%3B+expires%3DTue,+16-Jun-2020+01:53:53+GMT
4. When you open it, you will receive a browser cookie GOOGLE_ABUSE_EXEMPTION with the same data as the URL you have opened previously.
After that you will be 302 redirected to initial page https://www.google.com/search?q=kurwa
5. You open https://www.google.com/search?q=kurwa with GOOGLE_ABUSE_EXEMPTION cookie. And then you are getting your search results.

That is how it works. Now what we propose. We decided to go further and not only we solve captcha but also communicate with https://www.google.com/sorry/index

We've made two different methods to bypass a captcha. Both suppose that you will use your own proxy servers.

Method 1. GOOGLE_ABUSE address
Send us
pageurl - full URL of the page you are bypassing a captcha on
proxy - your proxy server address
googlekey=6LfwuyUTAAAAAOAmoS0fdqijC2PbbdH4kjq62Y1b
test=v

data-s is NOT needed
After resolution we will provide an URL contains
google_abuse=GOOGLE_ABUSE_EXEMPTION parameter
You have to open it, receive a cookie, execute a redirect and then you'll get your search results.

Method 2, a new Q with valid g-recaptcha-responce
Send us
pageurl - full URL of the page you are bypassing a captcha on
proxy - your proxy server address
googlekey=6LfwuyUTAAAAAOAmoS0fdqijC2PbbdH4kjq62Y1b
test=r

data-s is NOT needed
After resolution we will provide
g-recaptcha-responce: recaptcha token
q: new value, NOT from your pageurl
You have to perform a Step 2 actions, providing g-recaptcha-responce and q values.


What do both methods have in common
1. Only works using your proxy servers. Both Q and google_abuse are binded to your IP address. You've got to use the same proxy server to bypass a resolution which you send to us to get captcha solved.
2. Both methods are experimental. We can't guarantee a stable work yet.
3. In future we could change it and probably will only leave one of it as a main method.
 

Omarcoder

New member
Hello Petr, I'm trying with the method #1 (Using test=v), I made 4 different requests using 4 different proxies and on all of them I got "ERROR_CAPTCHA_UNSOLVABLE", is this intended behavior? maybe there is something I'm missing?
This is my URL, obviously using proper APIKEY and proxy (The proxy is in the same format I use to make the requests on my server using CURL, so I know it's good, and actually It was working before without those new methods but I'm getting too much "ERROR_CAPTCHA_UNSOLVABLE" responses)
Code:
https://2captcha.com/in.php?key={MYAPIKEY}&method=userrecaptcha&googlekey=6LfwuyUTAAAAAOAmoS0fdqijC2PbbdH4kjq62Y1b&pageurl=https%3A%2F%2Fwww.google.com%2Fsorry%2Findex%3Fcontinue%3Dhttps%3A%2F%2Fwww.google.com%2Fsearch%253Fq%253DDownload%252Bdiscord%26q%3DEgRfVR-DGKGOt_gFIhkA8aeDS87bVi1MeBk3hVx0bd8mvlwakNxOMgFy&proxytype=https&json=1&test=v&proxy=PROXYUSER:PROXYPASS@PROXYIP:PROXYPORT
 

sonsoixam

New member
1594741133551.png


Code:
https://www.google.com/sorry/index?continue=https://www.google.com.vn/search%3Fei%3DTdENX4fmM4O5rQGF3JHYCQ%26q%3Dallintitle%253Adi%2Bhoc%2Bthem%26oq%3Dallintitle%253Adi%2Bhoc%2Bthem%26gs_lcp%3DCgZwc3ktYWIQDFAAWABgswFoAHAAeACAAQCIAQCSAQCYAQCqAQdnd3Mtd2l6%26sclient%3Dpsy-ab%26ved%3D0ahUKEwjHwOi3ic3qAhWDXCsKHQVuBJsQ4dUDCAw&q=EgRzTxliGNCit_gFIhkA8aeDSxElK1NA4INLKEVPqeBaMykok7-sMgFy

i cant find that param in url , i use "allintitle:keyword" in query.
i also try to javascript to submit , but nothing happen , url reloads , no redirect

my javascript
  1. document.getElementById("g-recaptcha-response").innerHTML="TOKEN_FROM_2CAPTCHA";

  2. myCallbackFunction();

    Before , i use this method and success
 
Last edited:

Omarcoder

New member
Hi!
We'll investigate the issue and post an update here.

At the moment you can use the approach with data-s parameter.
Thanks for your response @Mark Miller , is there any success rate using data-s parameter? I'm experiencing an issue: When I send the g-recaptcha-response, it answers me another HTTP 429 code, a page like the one where the captcha is requested but that one doesn't have any captcha to solve.
I just did a test (Using data-s parameter), and my results are as following with 48 captchas required (Different IP's, cookies and User Agent):
7 - CAPTCHA UNSOLVABLE
3 - Solutions that returned 429 without the option to solve captcha
38 - Captchad solved

Right now I'm running a new test with more IP's, I would update numbers but I would like to know if there is some kind of percents to compare my results with.
 

Mark Miller

2Captcha Engineer
Staff member
Thanks for your response @Mark Miller , is there any success rate using data-s parameter? I'm experiencing an issue: When I send the g-recaptcha-response, it answers me another HTTP 429 code, a page like the one where the captcha is requested but that one doesn't have any captcha to solve.
I just did a test (Using data-s parameter), and my results are as following with 48 captchas required (Different IP's, cookies and User Agent):
7 - CAPTCHA UNSOLVABLE
3 - Solutions that returned 429 without the option to solve captcha
38 - Captchad solved

Right now I'm running a new test with more IP's, I would update numbers but I would like to know if there is some kind of percents to compare my results with.

Your numbers look pretty fine. The main problem of this method is it's really sensitive to IP address. The best success rate is achieved if you change the IP address after Google declines a token or after you receive UNSOLVABLE. In most cases that happens if Google temporary blocked the IP address.
 
Status
Not open for further replies.