File: //usr/lib64/python2.7/site-packages/mercurial/keepalive.pyc
�
1�3\c @@ s� d Z d d l m Z m Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l m
Z
d d l m Z m
Z
m Z m Z d d l m Z e j Z e j Z e j Z d a d e f d � � YZ d
e f d � � YZ d e e j f d
� � YZ d e j f d � � YZ d � Z d � Z d e j f d � � YZ d � Z d � Z! d d � Z" d � Z# d d � Z$ e% d k r�d d l& Z& y$ e' e j( d � Z) e j( d Z* Wn, e+ e, f k
r�e- d e j( d � q�Xe$ e* e) � n d S( sK An HTTP handler for urllib2 that supports HTTP 1.1 and keepalive.
>>> import urllib2
>>> from keepalive import HTTPHandler
>>> keepalive_handler = HTTPHandler()
>>> opener = urlreq.buildopener(keepalive_handler)
>>> urlreq.installopener(opener)
>>>
>>> fo = urlreq.urlopen('http://www.python.org')
If a connection to a given host is requested, and all of the existing
connections are still in use, another connection will be opened. If
the handler tries to use an existing connection but it fails in some
way, it will be closed and removed from the pool.
To remove the handler, simply re-run build_opener with no arguments, and
install that opener.
You can explicitly close connections by using the close_connection()
method of the returned file-like object (described below) or you can
use the handler methods:
close_connection(host)
close_all()
open_connections()
NOTE: using the close_connection and close_all methods of the handler
should be done with care when using multiple threads.
* there is nothing that prevents another thread from creating new
connections immediately after connections are closed
* no checks are done to prevent in-use connections from being closed
>>> keepalive_handler.close_all()
EXTRA ATTRIBUTES AND METHODS
Upon a status of 200, the object returned has a few additional
attributes and methods, which should not be used if you want to
remain consistent with the normal urllib2-returned objects:
close_connection() - close the connection to the host
readlines() - you know, readlines()
status - the return status (i.e. 404)
reason - english translation of status (i.e. 'File not found')
If you want the best of both worlds, use this inside an
AttributeError-catching try:
>>> try: status = fo.status
>>> except AttributeError: status = None
Unfortunately, these are ONLY there if status == 200, so it's not
easy to distinguish between non-200 responses. The reason is that
urllib2 tries to do clever things with error codes 301, 302, 401,
and 407, and it wraps the object upon return.
i ( t absolute_importt print_functionNi ( t _( t nodet pycompatt urllibcompatt util( t procutilt ConnectionManagerc B@ sG e Z d Z d � Z d � Z d � Z d � Z d � Z d d � Z RS( sV
The connection manager must be able to:
* keep track of all existing
c C@ s. t j � | _ i | _ i | _ i | _ d S( N( t threadingt Lockt _lockt _hostmapt _connmapt _readymap( t self( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt __init__s s c C@ ss | j j � zQ | | j k r/ g | j | <n | j | j | � | | j | <| | j | <Wd | j j � Xd S( N( R t acquireR t appendR
R t release( R t hostt
connectiont ready( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt addy s
c C@ s� | j j � zk y | j | } Wn t k
r4 nC X| j | =| j | =| j | j | � | j | sw | j | =n Wd | j j � Xd S( N( R R R
t KeyErrorR R t removeR ( R R R ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR � s
c C@ s) y | | j | <Wn t k
r$ n Xd S( N( R R ( R R R ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt set_ready� s
c C@ sz d } | j j � zR | | j k rd x< | j | D]* } | j | r3 d | j | <| } Pq3 q3 Wn Wd | j j � X| S( Ni ( t NoneR R R R R ( R R t connt c( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt get_ready_conn� s
c C@ s0 | r t | j j | g � � St | j � Sd S( N( t listR t gett dict( R R ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt get_all� s N(
t __name__t
__module__t __doc__R R R R R R R" ( ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR n s t KeepAliveHandlerc B@ sh e Z d d � Z d � Z d � Z d � Z d � Z d d � Z d � Z d � Z
d � Z d
� Z RS( c C@ s+ t � | _ | | _ d | _ d | _ d S( Ni ( R t _cmt _timeoutt
requestscountt sentbytescount( R t timeout( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR � s c C@ s8 g | j j � j � D] \ } } | t | � f ^ q S( st return a list of connected hosts and the number of connections
to each. [('foo.com:80', 2), ('bar.org', 1)]( R' R" t itemst len( R R t li( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt open_connections� s c C@ s; x4 | j j | � D] } | j j | � | j � q Wd S( s� close connection(s) to <host>
host is the host:port spec, as in 'www.cnn.com:8080' as passed in.
no error occurs if there is no connection to that host.N( R' R" R t close( R R t h( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt close_connection� s c C@ sU xN | j j � j � D]7 \ } } x( | D] } | j j | � | j � q) Wq Wd S( s close all open connectionsN( R' R" t iteritemsR R0 ( R R t connsR1 ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt close_all� s "
c C@ s | j j | d � d S( se tells us that this request is now closed and that the
connection is ready for another requesti N( R' R ( R t requestR R ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt _request_closed� s i c C@ s' | r | j � n | j j | � d S( N( R0 R' R ( R R R R0 ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt _remove_connection� s
c C@ s | j t | � S( N( t do_opent HTTPConnection( R t req( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt http_open� s c C@ s t j | � } | s' t j d � � n y� | j j | � } x� | r� | j | | | � } | rd Pn | j � | j j | � | j j | � } q? W| | d | j �} t
r� t
j d | t | � � n | j j
| | d � | j | | � | j � } Wnh t j k
r;} t j t d � t j | j � � � n. t j t j f k
rh} t j | � � n Xt | d t � r�| j j | � n t
r�t
j d | j | j � n | | _ | | _ | j � | _ | | _! | j | _" | j# | _$ | j | _# | S( Ns
no host givenR+ s" creating new connection to %s (%d)i s bad HTTP status line: %st
will_closes STATUS: %s, %s(% R t gethostt urlerrt urlerrorR' R t _reuse_connectionR0 R R( t DEBUGt infot idR t _start_transactiont getresponset httplibt
BadStatusLineR R t sysbytest linet sockett errort
HTTPExceptiont getattrt Truet statust reasont _handlert _hostt get_full_urlt _urlt _connectiont codet msgt headers( R t
http_classR; R R1 t rt err( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR9 � sH
" c C@ s� y | j | | � | j � } Wnh t j t j f k
rE d } nF t rj t j d | t | � � n | j j
| � | j � � n X| d k s� | j d k r� t r� t j
d | t | � � n d } n" t r� t j
d | t | � � n | S( sG start the transaction with a re-used connection
return a response object (r) upon success or None on failure.
This DOES not close or remove bad connections in cases where
it returns. However, if an unexpected exception occurs, it
will close and remove the connection before re-raising.
s4 unexpected exception - closing connection to %s (%d)i s& failed to re-use connection to %s (%d)s re-using connection to %s (%d)N( RE RF RK RL RG RM R RB RD R' R R0 t versionRC ( R R1 R; R R[ ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyRA s(
c C@ sk t | d d � } t j | j j � } | j t | j j � � � | j t | j j � � � t j d � | j � D� � } i } x7 d D]/ } | | k r� d | d | j
d d � <q� q� Wy� t j | � rLt j
| � } | j | j � t j | � | � d
| k r | j d d � n d
| k rn| j d d t | � � qnn" | j | j � t j | � | � Wn% t j k
r�} t j | � � n Xx* | j � D] \ } }
| j | |
� q�W| j � t j | � r�| j | � n | j d 7_ | j t | d d � | 7_ y8 | j j d 7_ | j j t | d d � | 7_ Wn t k
rfn Xd S( NR* i c s@ s' | ] \ } } | j � | f Vq d S( N( t lower( t .0t nt v( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pys <genexpr>F s R s accept-encodingi t skip_t -R s content-types Content-types! application/x-www-form-urlencodeds content-lengths Content-lengths %d( R s accept-encoding( RN R t sortdictt parentt
addheaderst updatet sortedRY R, t unredirected_hdrst replaceR t hasdatat getdatat
putrequestt
get_methodt getselectort putheaderR- RK RL R? R@ t
endheaderst sendR) R* t AttributeError( R R1 R; t
oldbytescountRY t skipheadersR` t dataR\ t kRa ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyRE = sL
!
N(
R# R$ R R R/ R2 R5 R7 R8 R< R9 RA RE ( ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR& � s 7 .t HTTPHandlerc B@ s e Z RS( ( R# R$ ( ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyRx m s t HTTPResponsec B@ s� e Z d d d d � Z e j j Z e e j d d � Z d � Z
d � Z d � Z d � Z
d d � Z d � Z d � Z d d
� Z d � Z RS(
i c C@ s� i } t j s& t | d <t | d <n t j j | | d | d | | �| j | _ d | _ d | _ d | _
d | _ d | _ d | _
d | _ d | _ d S( Nt strictt bufferingt
debuglevelt methodi t i� ( R t ispy3RO RG Ry R t filenoR RW t receivedbytescountt _rbuft _rbufsizeRR RS RU RV ( R t sockR| Rz R} t extrakw( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR � s
t readintoc C@ sN | j rJ | j j � d | _ | j rJ | j j | | j | j � qJ n d S( N( t fpR0 R RR R7 RS RV ( R ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR0 � s
c C@ s- | j j | j | j d d �| j � d S( NR0 i ( RR R8 RS RV R0 ( R ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR2 � s c C@ s | j S( N( RY ( R ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyRC � s c C@ s | j S( N( RU ( R ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt geturl� s c C@ s | j ra | d k ra t | j � } | | k r= | | 8} qa | j | } | j | | _ | Sn | j } d | _ | j | � } | j t | � 7_ y | j j t | � 7_ Wn t k
r� n Xy | j j j t | � 7_ Wn t k
r� n X| | 7} | S( NR~ ( R� R R- t _raw_readR� RV Rs RR Re ( R t amtt Lt sRv ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt read� s*
c C@ s� | j } g } x�t r�| d k r� | j j � } | j d � } | d k r[ | | } n y t | d � } Wn3 t k
r� | j � t j
d j | � � � n X| d k r� Pq� n | d k r� | j | j
| � � n� | | k r| j | j
| � � | | | _ d j | � S| | k r]| j | j
| � � | j
d � d | _ d j | � S| j | j
| � � | | 8} | j
d � d } q Wx3 t r�| j j � } | s�Pn | d k r�Pq�q�W| j � d j | � S( Nt ;i i R~ i s
( t
chunk_leftRO R R� t readlinet findt intt
ValueErrorR0 RG t IncompleteReadt joinR t
_safe_read( R R� R� t partsRJ t i( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt
_read_chunked� sL
c C@ sr | j j d � } | d k rI | d 7} | j | } | j | | _ | S| j g } d } | j } x� t r| j | � } | s� Pn | j t | � 7_ | j j t | � 7_ y | j j j t | � 7_ Wn t
k
r� n X| j | � | j d � } | d k rg Pqg qg W| d k r8d | _ d j | � S| d 7} | d | | _ | d | | d <d j | � S( Ns
i i i����R~ (
R� R� R� RO R� R� R- RV RR Re Rs R R� ( R R� RJ t chunkst readsizet new( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR� s<
c C@ sf d } g } xS t ra | j � } | s+ Pn | j | � | t | � 7} | r | | k r Pq q W| S( Ni ( RO R� R R- ( R t sizehintt totalR RJ ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt readlines* s
c C@ s7 | j d k rK | j t | � � } | s. d S| | d t | � +t | � St | � } t | j � } | | k r� | j | | d | +| j | | _ | St | � } | j | | | !� } | j | 7_ | j j | 7_ y | j j | 7_ Wn t k
rn X| j | d | +| t | j � 7} d | _ | S( Ni R~ (
t
_raw_readintoR R� R- R� t
memoryviewR� RV RR Rs ( R t destt resR� t havet mvt got( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR� 7 s0
N( R# R$ R R RG Ry R� R� RN R� R0 R2 RC R� R� R� R� R� ( ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyRy p s 6 *
c C@ s� t | d d � d k r d S| j d k rM | j rA | j � qM t j � n | j d k rr t d t | � � n y� d } t | d d � } | d k r| j d k r� t d � n | | � } xc | r� | j j
| � | j t | � 7_ | | � } q� Wn% | j j
| � | j t | � 7_ Wn{ t
j k
r�} t } | d t j k r�| j t j k r�d | _ | j � | _ t } n | j � n | r�� q�n Xd S( s` Send `str' to the server.
Shamelessly ripped off from httplib to patch a bad behavior.
t _broken_pipe_respNi s send:i R� s sending a read()able( RN R R� t auto_opent connectRG t NotConnectedR| t printt reprt sendallR* R- RK RL RO t errnot EPIPEt _HTTPConnection__statet _CS_REQ_SENTR� RF t FalseR0 ( R t strt blocksizeR� Rv Ra t reraise( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt safesendS s>
c @ s" � f d � } � j j | _ | S( s> Wraps getresponse in cls with a broken-pipe sane version.
c @ s/ t | d d � } | d k r" | S� j | � S( NR� ( RN R RF ( R R[ ( t cls( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt safegetresponse� s ( RF R% ( R� R� ( ( R� s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt wrapgetresponse� s R: c B@ s, e Z e Z e Z e e j � Z d � Z
RS( c O@ s, t j j | | | � d | _ d | _ d S( Ni ( RG R: R R* R� ( R t argst kwargs( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR � s ( R# R$ Ry t response_classR� Rr R� RG R: RF R ( ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR: � s c C@ sn t j } d } t j � } t j | � t j | � } | j � } | j � | | � } t | d t j
| j � � f � t j t � � } t j | � t j | � } | j � } | j � | | � } t | d t j
| j � � f � t j | � } d } x* t
r0| j � } | r,| | } qPqW| j � | | � } t | d t j
| j � � f � d S( Ns %25s: %ss
normal urllibs keepalive readR~ s keepalive readline( t hashlibt md5t urlreqt buildopenert
installopenert urlopenR� R0 R� R t hext digestRx RO R� ( t urlR� t formatt openert fot foot mt f( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt
continuity� s4
#
#
c C@ s� t d | | f � t j j d � t j � } t j | � t | | � } t d | � t j j d � t j t � � } t j | � t | | � } t d | � t d | | � d S( Ns making %i connections to:
%ss( first using the normal urllib handlerss TIME: %.3f ss( now using the keepalive handler s improvement factor: %.2f( R� R t stdoutt writeR� R� R� t fetchRx ( t NR� R� t t1t t2( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt comp� s
c C@ s� d d l } g } | j � } xn t | � D]` } | rS | d k rS | j | � n t j | � } | j � } | j � | j t | � � q+ W| j � | } d }
xC | d D]7 } |
d }
| | d k s� t d |
| f � q� q� W| S( Ni i s+ WARNING: inconsistent length on read %i: %i(
t timet ranget sleepR� R� R� R0 R R- R� ( R� R� t delayR� t lenst starttimeR� R� R� t difft j( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR� � s"
c C@ s t } d t f d � � Y} | � a t d � t j | � } | j � } | j � d } t d | � xH | d k r� t j j d | � t j j
� t j d � | d 8} qk Wt j
j d � t d
� t j | � } | j � } | j � | | k r t d � n
t d � | a d S(
Nt
FakeLoggerc B@ s e Z d � Z e Z Z Z RS( c W@ s t | | � d S( N( R� ( R RX R� ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt debug� s ( R# R$ R� RC t warningRL ( ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyR� � s s- fetching the file to establish a connectioni s; waiting %i seconds for the server to close the connectioni s
%2ii s
s! fetching the file a second times data are identicals ERROR: DATA DIFFER( RB t objectR� R� R� R� R0 R R� R� t flushR� R� t stderr( R� t dbbackupR� R� t data1R� t data2( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt test_timeout� s.
i
c C@ sW t d � t | � t d � t d � t | | � t d � t d � t | � d S( Ns> performing continuity test (making sure stuff isn't corrupted)R~ s performing speed comparisons# performing dropped-connection check( R� R� R� R� ( R� R� ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt test s
t __main__i s %s <integer> <url>(. R% t
__future__R R R� R� RK t sysR t i18nR R~ R R R R t utilsR RG R? R� R RB R� R R& t httphandlerRx Ry R� R� R: R� R� R� R� R� R# R� R� t argvR� R� t
IndexErrorR� R� ( ( ( s9 /usr/lib64/python2.7/site-packages/mercurial/keepalive.pyt <module>Q sB " @� � 5 # #