大家好,初学爬虫
最近需要在https://www.taiwanmobile.com/mobile/storelbs/lbs.html#
撷取店点相关资讯
爬之前的文章发现header似乎是问题所在
于是把header全部放进去,但好像没有用...
想请教各位大大该如何克服呢?感恩
程式码如下:
form_data = {"city":"台北市",
"district":"松山区",
"lat":"25.0464207",
"lng":"121.5555859",
"searchDistance":"-1"}
headers = {"Accept":"application/json, text/javascript, */*; q=0.01",
"Accept-Encoding":"gzip, deflate, br",
"Accept-Language":"zh-TW,zh;q=0.8,en-US;q=0.6,en;q=0.4",
"Connection":"keep-alive",
"Content-Length":"127",
"Content-Type":"application/x-www-form-urlencoded; charset=UTF-8",
"Cookie":"_msuuid_558dza12683=D58CE660-772F-4345-A7C6-B1B732FB85F0; \
JSESSIONID=nt+dIYPkeJGODFLtAALzUKsu; _ga=GA1.2.142560397.1498299011; \
_gid=GA1.2.594796640.1506128988",
"Host":"www.taiwanmobile.com",
"Origin":"https://www.taiwanmobile.com",
"Referer":"https://www.taiwanmobile.com/mobile/storelbs/lbs.html",
"User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWe\
bKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36",
"X-Requested-With":"XMLHttpRequest"}
response_post = requests.post("https://www.taiwanmobile.com/mobile/stor\
elbs/lbs.html#", data = form_data, headers=headers)
response_post.encoding = 'utf-8'
soup_post = BeautifulSoup(response_post.text, "lxml")