python - testing urllib2 application, http responses loaded from files -

My Python application makes several HTN requests for many URLs using urlib2. I would like to create a unit test suite to test my data parsing and error handling code.

I have a test-filled directory, with many files, each file has a single HTTP response headers and feedback data (using curl -i) in some cases, these files In order to change the urllib2.urlopen and return a fake, I would like to create a duplicate object

Ideally, P> I am thinking that The URLB2 file is an easy way to load an HTTP response directly and urlib2 is to parse this data to make the appropriate reaction object (such as the response was read from the URL.

I did "file" : // "Protocol, although the http response headers at the top of the file could not be read and were not parsed properly.

Alternatively I am Consider typing a small web server class to serve test files, although it seems like a little more work than I like. Urllib2 in any way responds to the object I have already saved in the http response Will it be easy to reorder from

Any ideas?

I think the best way to ridicule a subset of httplib.HTTPConnection (For the concreteness in the following, call the resulting square for mockcon ) and add a handler using it and sub-coding httpHandler ( build_opener To use in - sub-cling means that can replace HTTPHandler build_opener by default):

  class MockHTTPHandler (urllib2.HTTPHandler): def http_open ( Auto, request): Return self.do_open (mockcon, req)  

The methods for the call to the mockcans should be supplied by the code do_open - many dummies (ie) Accept and ignore uncontrolled ARG and KWDS and do not do anything):

  set_debuglevel _set_tunnel request  

( 2 of the request Arg can be interested in , because it divides the "selector" of the URL).

__init __ method mockcon receives the host part of the URL the first argument (i.e., first self of course Afterwards) and the following should be ignored by KWDS (used to set a timeout).

get_response method mockcon (any argument, beyond the curriculum) ) will have to return an HTTP response object - that is, a file Any readable object like .msg is the attribute , .status , and .reason , and a method get_full_url () URL to back

You can use an actual httplib.HTTPResponse example for the latter role, but you should start with a fake / dummy Razer that contains a Makefile argument is DKWD and whatever returns), and, immediately after starting it, file a .pp argument to an rb open Reset for, which actually provide bytes which will receive a real HTTP response to your Saw On freight.

I think that creating a complete motive for the urllib2.urlopen call can be simpler with this effort to reuse the functionality of this code. Urllib2 (and httplib which uses it internally), although probably not as simple as the "local web server" approach, which you think is more work. But it is worth noting in all three ways (fake will definitely be in the most light / fast operation, the local web server will slow down ... and modify the URL by prefixing anyway by http: // localhost Need to: Some ports / them, of course).