Metadata-Version: 2.1
Name: pifetcher
Version: 0.0.3.8
Summary: A scalable headless data fetching library written with python and message queue service to enable quickly and easily prasing web data in a distributive way.
Home-page: https://github.com/gavinz0228/pifetcher
Author: Gavin Zhang
Author-email: gavinz0228@gmail.com
License: UNKNOWN
Description: # pifetcher
        A scalable headless data fetching library written with python and message queue service to enable quickly and easily parsing web data in a distributive way.
        
        ## To install
        ```bash
        pip install pifetcher
        ```
        
        ## PYPI Link  [https://pypi.org/project/pifetcher/]
        
        ## dependencies:
        - pyppeteer
        - BeautifulSoup4
        - boto3
        
        ## features:
        
        - event-callback-based interaction between user defined logic and the pre-disigned fetch worker
        - process works in batches, library user will be able to capture the event of a batch of works have been finished
        - easy to use, only needs to inherit the FetchWorker class and implement the basic call back functions
        - it's design to use message queue, enbles more than just one worker to perform data fetching in order to scale the application 
        
        ## how to use:
        
        Github: [https://github.com/gavinz0228/pifetcher]
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX
Description-Content-Type: text/markdown
