?i hdZddlZddlZddlZddlZddlmZmZddlm Z m Z m Z m Z ddl mZdZdZd Zd d d d Zdddd ZdefdZdedefdZdedefdZdedefdZ d&dedededede ee ff dZde ee fde e ee ffd Zd'd!ed"ede ee ffd#Z d&d$e e ee fdede e ee ffd%Zy)(zHacker News search via Algolia API (free, no auth required). Uses hn.algolia.com/api/v1 for story discovery and comment enrichment. No API key needed - just HTTP calls via stdlib urllib. N)ThreadPoolExecutor as_completed)AnyDictListOptional)httpz$https://hn.algolia.com/api/v1/searchz,https://hn.algolia.com/api/v1/search_by_datez#https://hn.algolia.com/api/v1/items<)quickdefaultdeep msgctjjrBtjjd|dtjj yy)zHLog to stderr (only in TTY mode to avoid cluttering Claude Code output).z[HN]  N)sysstderrisattywriteflush)rs U/home/ubuntu/.openclaw/workspace/skills/last30days-official/scripts/lib/hackernews.py_logr!sA zz 5R) date_strreturnc |jd}t|dt|dt|d}}}ddl}ddl}|j||||jj }t|j S)z8Convert YYYY-MM-DD to Unix timestamp (start of day UTC).-rr N)tzinfo)splitintcalendardatetimetimezoneutc timestamp)rpartsyearmonthdayr'r(dts r _date_to_unixr1(sq NN3 E58}c%(mSq]%D   4H4E4E4I4I  JB r||~ rtscddl}|jj||jj}|j dS)z%Convert Unix timestamp to YYYY-MM-DD.rN)tzz%Y-%m-%d)r( fromtimestampr)r*strftime)r2r(r0s r _unix_to_dater72s;    ( (0A0A0E0E ( FB ;;z ""rtextcddl}tj|}|jdd|}|jdd|}|j S)z9Strip HTML tags and decode entities from HN comment text.rNz

rz<[^>]+>)rehtmlunescapesubstrip)r8r;s r _strip_htmlr@9sC == D 66&$ %D 66*b$ 'D ::<rtopic from_dateto_datedepthc Ntj|td}t|}t|dz}td|d|d|d|dd|d |t |d }d d lm}td ||} tjd| d} | jdg} tdt| d| S#tj$r&} td| gt | dcYd} ~ Sd} ~ wt$r&} td| gt | dcYd} ~ Sd} ~ wwxYw)aSearch Hacker News via Algolia API. Args: topic: Search topic from_date: Start date (YYYY-MM-DD) to_date: End date (YYYY-MM-DD) depth: 'quick', 'default', or 'deep' Returns: Dict with Algolia response (contains 'hits' list). riQzSearching for 'z ' (since z, count=)storyz created_at_i>z,created_at_i<)querytagsnumericFilters hitsPerPager) urlencode?GETr timeoutzSearch failed: )hitserrorNrQzFound z stories) DEPTH_CONFIGgetr1rstr urllib.parserLALGOLIA_SEARCH_URLr request HTTPError Exceptionlen) rArBrCrDcountfrom_tsto_tsparamsrLurlresponseerQs rsearch_hackernewsrcBs1"   UL$; >- qc "#SV,, - qc "#SV,,-s0:B>>D$C2,D$2 D$>DD$D$rac|jdg}g}t|D]#\}}|jdd}|jdxsd}|jdxsd}|jd}d} |r t|} |jd xsd} d |} td d |d zz } t dt j |dz } t d | dz| zdz}|j||jdd| | |jdd| ||dt|dd|jddddd &|S)zyParse Algolia response into normalized item dicts. Returns: List of item dicts ready for normalization. rQobjectIDr:pointsr num_comments created_at_iNr`z%https://news.ycombinator.com/item?id=g333333?g?g{Gz?g?(gffffff?g?titleauthor)rfrgr#zHN story about rAr ) object_idrjr`hn_urlrkdate engagement relevance why_relevant) rT enumerater7maxminmathlog1pappendround)rarQitemsihitrlrfrgrhr article_urlrm rank_scoreengagement_boostrps rparse_hackernews_responserrsi << #D ED/! 3GGJ+ "'aww~.3! ww~.  $\2Hggen* 8 DcQX./ sDJJv$6$;<Z#-0@@3FG  "WWWb)ggh+ ,y!,-cgggw.G.L-MN  )! F Lrrl max_commentsctd|} tjd|d}|j d g}|Dcgc]&}|j d r|j d r|(}}|j d d g}g} |d|D]}t|j d d} t| dkDr| dddzn| } |j|j d d| |j dxsdd| jddjdddd} | s| j| || dS#t$r }t d|d|ggdcYd}~Sd}~wwxYwcc}w)zFetch top-level comments for a story from Algolia items endpoint. Args: object_id: HN story ID max_comments: Max comments to return Returns: Dict with 'comments' list and 'comment_insights' list. /rNr rOzFailed to fetch comments for z: )commentscomment_insightsNchildrenr8rkc,|jdxsdS)NrfrrT)cs rz&_fetch_item_comments..sQUU8_%9rTkeyreverser:i,z...rfr)rkr8rfz. r) ALGOLIA_ITEM_URLr rXrZrrTsortr@r[rwr%) rlrr`datarbrr real_commentsrinsightsr8excerptfirst_sentences r_fetch_item_commentsrs a { +C8||E33 xx B'H 55=QUU8_ M94HHH =L ) ,155,-(+D C$t*u$TeeHb)eeHo*  D)!,2248;DSA  OON + ,!h ??9 8 ,YKr! =>B778s"D7+E#7 E EE E ryc (sStj|td}ttt fdd}|d|}t dt |dt d 5}|Dcic]}|jt|d |!}}t|D]0}||} |jd } | d |d<| d|d<2 dddScc}w#t$rg|d<g|d<Y^wxYw#1swYSxYw)zFetch comments for top N stories by points. Args: items: Parsed HN items depth: Research depth (controls how many to enrich) Returns: Items with top_comments and comment_insights added. rcN|jdijddS)Nrorfrr)rzrys rrz$enrich_top_stories..s#eAhll<4881ErTrNzEnriching top z stories with commentsr) max_workersrlr rOr top_commentsr) ENRICH_LIMITSrTsortedranger[rrsubmitrrresultrZ) ryrDlimit by_points to_enrichexecutoridxfuturesfuturers ` renrich_top_storiesrsa    e]9%= >E c%j EI &5!I>#i.))? @A  *4h !   OO$c ;'    #7+ 4F&/C 4r2-3J-?c >*178J1Kc -.  44& L%  4-/c >*13c -. 44& LsB/D4$C#D.(C(D#D(DDDDD)r)r)__doc__r<rurtimeconcurrent.futuresrrtypingrrrrr:r rWALGOLIA_SEARCH_BY_DATE_URLrrSrrUrr&r1r7r@rcrrrrrrsn  ?,,;K8    cCC#c#c#cc - ---  -  #s(^ -`,S#X,4S#X;O,^*@C*@s*@4S>*@^/ S#X / / $sCx./r