[问题] 网页爬虫问题

楼主: ntasop (kuli)   2017-11-11 12:28:29
请问大家用post方式抓网页资料,同样方式加request header和json资料,
python可以正常抓到网络资料,但C#没办法,请问大家有什么地方还需要
注意,非常谢谢。
C#版本:
HttpClient httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
httpClient.DefaultRequestHeaders.Add("referer", "https://shopee.tw/jeanch911");
httpClient.DefaultRequestHeaders.Add("user-agent", "Chrome/61.0.3163.100 Safari/537.36");
httpClient.DefaultRequestHeaders.Add("x-csrftoken", "iGwDsBPnVwMlr68yxp23vxYB3N53gMRH");
httpClient.DefaultRequestHeaders.Add("cookie", "略过太长....");
httpClient.BaseAddress = new Uri("https://shopee.tw/");
string str = "{\"shop_ids\":[6550848]};
JObject jsonObject = JObject.Parse(str);
var content = new StringContent(jsonObject.ToString(), Encoding.UTF8, "application/json");
var respone = await httpClient.PostAsync("api/v1/shops", content);
System.Console.WriteLine(respone.Content.ReadAsStringAsync().Result);
python版本:
headers = {
'Cookie':'.......略过太长',
'Referer':'https://shopee.tw/trc.tw',
'User-Agent':'Chrome/61.0.3163.100 Safari/537.36',
'X-CSRFToken':'3EwwiZKHrh74ZjKB1oHycwOgS6COQNon'}
jd = json.loads('{"shop_ids":[24550283]}')
response = requests.post('https://shopee.tw/api/v1/shops/', json = jd, headers = headers)
print(response.text.encode('utf-8').decode('unicode_escape'))
作者: vi000246 (Vi)   2017-11-11 13:22:00
抓封包看python跟c#有什么不同
楼主: ntasop (kuli)   2017-11-11 16:52:00
请问vi000246大,因没学过这部分知识,该如何抓封包?
作者: james732 (好人超)   2017-11-11 18:15:00
用wireshark之类的工具吧?
作者: name2name2 (yang~hi)   2017-11-12 10:49:00
用httpclient.PostAsJsonAsync 试试
作者: vi000246 (Vi)   2017-11-12 20:08:00
chrome的F12、Fiddler、wireshark
作者: groovy2016   2017-11-13 01:44:00
firebug postman

Links booklink

Contact Us: admin [ a t ] ucptt.com