­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­  ?J[c@sodZdZddlZddlmZddlmZddlZddlmZmZddlm Z ddl Z ddl Z ddl Z ddl Z ddlZddlZddlZddlZdZed Zd efd YZd Zd ZdZddZddZddZddZdddZedkrkeejjndS(s=Diagnostic functions, mainly for use when doing tech support.tMITiN(tStringIO(t HTMLParser(t BeautifulSoupt __version__(tbuilder_registryc CsVdtGHdtjGHdddg}xK|D]C}x:tjD]}||jkr;Pq;q;W|j|d|GHq+Wd|kr|jdy2dd lm }d d j t t |j GHWqtk r}d GHqXnd|kr!ydd l}d|jGHWq!tk r}dGHq!Xnt|dr?|j}n|jds]|jdrod|GHdGHd SyCtjj|rd|GHt|}|j}Wd QXnWntk rnXHx|D]}d|GHt} yt|d|} t} Wn&tk r'}d|GHtjnX| rEd|GH| jGHnddGHqWd S(s/Diagnostic suite for isolating common problems.s'Diagnostic running on Beautiful Soup %ssPython version %ss html.parserthtml5libtlxmls;I noticed that %s is not installed. Installing it may help.slxml-xmli(tetreesFound lxml version %st.s.lxml is not installed or couldn't be imported.NsFound html5lib version %ss2html5lib is not installed or couldn't be imported.treadshttp:shttps:s<"%s" looks like a URL. Beautiful Soup is not an HTTP client.spYou need to use some other library to get the document behind the URL, and feed that document to Beautiful Soup.s7"%s" looks like a filename. Reading data from the file.s#Trying to parse your markup with %stfeaturess%s could not parse the markup.s#Here's what %s did with the markup:t-iP(RtsystversionRtbuildersR tremovetappendRRtjointmaptstrt LXML_VERSIONt ImportErrorRthasattrR t startswithtostpathtexiststopent ValueErrortFalseRtTruet Exceptiont tracebackt print_exctprettify( tdatat basic_parserstnametbuilderRteRtfptparsertsuccesstsoup((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pytdiagnosesd       "            cKs[ddlm}xD|jt|d||D]$\}}d||j|jfGHq/WdS(sPrint out the lxml events that occur during parsing. This lets you see how lxml parses a document when no Beautiful Soup code is running. i(Rthtmls %s, %4s, %sN(RRt iterparseRttagttext(R$R.tkwargsRteventtelement((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pyt lxml_traceZs+tAnnouncingParsercBsheZdZdZdZdZdZdZdZdZ dZ d Z d Z RS( s?Announces HTMLParser parse events, without doing anything else.cCs |GHdS(N((tselfts((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pyt_pgscCs|jd|dS(Ns%s START(R9(R7R&tattrs((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pythandle_starttagjscCs|jd|dS(Ns%s END(R9(R7R&((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pyt handle_endtagmscCs|jd|dS(Ns%s DATA(R9(R7R$((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pyt handle_datapscCs|jd|dS(Ns %s CHARREF(R9(R7R&((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pythandle_charrefsscCs|jd|dS(Ns %s ENTITYREF(R9(R7R&((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pythandle_entityrefvscCs|jd|dS(Ns %s COMMENT(R9(R7R$((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pythandle_commentyscCs|jd|dS(Ns%s DECL(R9(R7R$((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pyt handle_decl|scCs|jd|dS(Ns%s UNKNOWN-DECL(R9(R7R$((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pyt unknown_declscCs|jd|dS(Ns%s PI(R9(R7R$((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pyt handle_pis( t__name__t __module__t__doc__R9R;R<R=R>R?R@RARBRC(((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pyR6ds         cCst}|j|dS(sPrint out the HTMLParser events that occur during parsing. This lets you see how HTMLParser parses a document when no Beautiful Soup code is running. N(R6tfeed(R$R*((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pythtmlparser_traces taeioutbcdfghjklmnpqrstvwxyzicCsSd}xFt|D]8}|ddkr2t}nt}|tj|7}qW|S(s#Generate a random word-like string.tii(tranget _consonantst_vowelstrandomtchoice(tlengthR8titt((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pytrwords icCsdjdt|DS(s'Generate a random sentence-like string.t css'|]}ttjddVqdS(ii N(RTROtrandint(t.0RR((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pys s(RRL(RQ((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pyt rsentencesicCsdddddddg}g}xt|D]}tjdd }|dkrutj|}|jd |q.|d kr|jttjd d q.|d kr.tj|}|jd|q.q.Wddj|dS(s+Randomly generate an invalid HTML document.tptdivtspanRRtbtscriptttableiis<%s>iiisss s(RLRORVRPRRXR(t num_elementst tag_namestelementsRRRPttag_name((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pytrdocs  " ic CsSdtGHt|}dt|GHxdddgddgD]}t}y1tj}t||}tj}t}Wn&tk r}d|GHtj nX|r=d|||fGHq=q=Wd d l m }tj}|j |tj}d ||GHd d l } | j}tj}|j|tj}d ||GHd S(s.Very basic head-to-head performance benchmark.s1Comparative parser benchmark on Beautiful Soup %ss3Generated a large invalid HTML document (%d bytes).RR.Rs html.parsers%s could not parse the markup.s"BS4+%s parsed the markup in %.2fs.i(Rs$Raw lxml parsed the markup in %.2fs.Ns(Raw html5lib parsed the markup in %.2fs.(RRctlenRttimeRRR R!R"RRtHTMLRRtparse( R_R$R*R+taR,R\R(RR((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pytbenchmark_parserss4               RcCstj}|j}t|}tdtd|d|}tjd|||tj |}|j d|j dddS(Ntbs4R$R*sbs4.BeautifulSoup(data, parser)t cumulatives _html5lib|bs4i2( ttempfiletNamedTemporaryFileR&RctdictRjtcProfiletrunctxtpstatstStatst sort_statst print_stats(R_R*t filehandletfilenameR$tvarststats((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pytprofiles    t__main__( RFt __license__RoRRRjRRt bs4.builderRRRqRORlReR!R R-RR5R6RHRNRMRTRXRcRiRyRDtstdinR (((s0/usr/lib/python2.7/site-packages/bs4/diagnose.pyts8           C !