Metadata-Version: 1.1
Name: keras-bert
Version: 0.7.0
Summary: BERT implemented in Keras
Home-page: https://github.com/CyberZHG/keras-bert
Author: CyberZHG
Author-email: CyberZHG@gmail.com
License: MIT
Description: 
        Keras BERT
        ==========
        
        
        .. image:: https://travis-ci.org/CyberZHG/keras-bert.svg
           :target: https://travis-ci.org/CyberZHG/keras-bert
           :alt: Travis
        
        
        .. image:: https://coveralls.io/repos/github/CyberZHG/keras-bert/badge.svg?branch=master
           :target: https://coveralls.io/github/CyberZHG/keras-bert
           :alt: Coverage
        
        
        Implementation of the paper: `BERT: Pre-training of Deep Bidirectional Transformers for
        Language Understanding <https://arxiv.org/pdf/1810.04805.pdf>`_
        
        Install
        -------
        
        .. code-block:: bash
        
           pip install keras-bert
        
        Usage
        -----
        
        .. code-block:: python
        
           from keras_bert import get_base_dict, get_model, gen_batch_inputs
        
        
           sentence_pairs = [
               [['all', 'work', 'and', 'no', 'play'], ['makes', 'jack', 'a', 'dull', 'boy']],
               [['from', 'the', 'day', 'forth'], ['my', 'arm', 'changed']],
               [['and', 'a', 'voice', 'echoed'], ['power', 'give', 'me', 'more', 'power']],
           ]
           token_dict = get_base_dict()
           for pairs in sentence_pairs:
               for token in pairs[0] + pairs[1]:
                   if token not in token_dict:
                       token_dict[token] = len(token_dict)
           token_list = list(token_dict.keys())
           model = get_model(
               token_num=len(token_dict),
               head_num=5,
               transformer_num=12,
               embed_dim=25,
               feed_forward_dim=100,
               seq_len=20,
               pos_num=20,
               dropout=0.05,
           )
           model.summary()
        
           def _generator():
               while True:
                   yield gen_batch_inputs(
                       sentence_pairs,
                       token_dict,
                       token_list,
                       seq_len=20,
                       mask_rate=0.3,
                       swap_sentence_rate=1.0,
                   )
        
           model.fit_generator(
               generator=_generator(),
               steps_per_epoch=1000,
               epochs=100,
               validation_data=_generator(),
               validation_steps=100,
               callbacks=[
                   keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)
               ],
           )
        
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3.6
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
