Archived Forum Post

Index of archived forum posts

Question:

Google scrape issue

May 13 '16 at 10:14

I'm trying scrape google results useing http object in VBScript.

here is the code:

set http = CreateObject("Chilkat_9_5_0.Http")
success = http.UnlockComponent("xxxx.my.key.goes.here.xxx")

deleteFile curpath & "page.html"
deleteFile curpath & "httpSessionLog.txt"
deleteFile curpath & "urls.csv"

'http.FollowRedirects = 1
http.ProxyDomain = "xxx.xxx.xxx.xxx"    
http.ProxyPort = "xxxx"
http.MimicFireFox = "1"
'http.SessionLogFilename = curpath & "httpSessionLog.txt"
http.CookieDir = "memory"
http.SaveCookies = 1
http.SendCookies = 1

url = "https://www.google.com/webhp?#gl=us&hl=en&q=what+is+my+ip+address"
html = http.QuickGetStr(url)

I know google does redirects as well drops some cookies, but can't figure out how to get actual results. I keep geting initial google page with search box and no results.

Please help

---

Answer

Maybe try with urlEncode?

---

Answer

RootTag it want do much difference, if you copy and paste the url in your browser this will load the results properly. this is actually the right way to replace spaces with pluses