16 Jan
2013
16 Jan
'13
2:57 a.m.
On 16/01/13 11:12, Brian May wrote:
On 16 January 2013 10:28, Trent W. Buck <trentbuck@gmail.com <mailto:trentbuck@gmail.com>> wrote:
Geoff D'Arcy wrote: > Most pay-walled News.com articles are accessible in full via Google.
If so, then tell your browser --user-agent='Hi I'm the booblegot' or whatever their spider sends, and you should be golden.
My (untested) theory is that is looks at the Referer: header, not the user-agent header.
Yes I suspect the same thing, I'll test with a debug proxy next time I'm using one.