Metadata-Version: 2.1
Name: structlog-extensions-nralbers
Version: 1.0.4
Summary: Processors for Structlog library
Home-page: https://structlog-extensions-nralbers.readthedocs.io/en/latest/
Author: Niels Albers
Author-email: nralbers@gmail.com
License: MIT
Project-URL: Documentation, https://structlog-extensions-nralbers.readthedocs.io/en/latest/
Project-URL: Source, https://github.com/nralbers/structlog-extensions-nralbers
Project-URL: Tracker, https://github.com/nralbers/structlog-extensions-nralbers/issues
Keywords: logging,structured,structure,log,ecs,apache-combined-log
Platform: UNKNOWN
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 5 - Production/Stable
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Intended Audience :: Developers
Requires-Python: ~=3.5
Description-Content-Type: text/x-rst
License-File: LICENSE

==========================================================================
``structlog-extensions-nralbers``:  Extension processors for ``structlog``
==========================================================================

.. -begin-short-

Structlog extensions are a set of `structlog <http://www.structlog.org/en/stable/>`_ processors and utility functions
to add new logging options to structlog. The primary purpose is to supply tools to convert existing structlog
logging output into `Elastic Common Schema <https://www.elastic.co/guide/en/ecs/current/index.html>`_
json output so users can easily plug their application output into centralised logging solutions such as
`ELK stack <https://www.elastic.co/what-is/elk-stack>`_.

At present the extensions consist of a ``CombinedLogParser`` (which for example can be used to convert gunicorn access log
output into ECS fields), and the ``NestedDictJSONRenderer``, which can be used to convert the output of the CombinedLogParser
into ECS json format output.

.. -end-short-

Usage
=====

.. -begin-usage-

``CombinedLogParser``
---------------------

This processor will parse events formatted in Apache Combined log format into
Elastic common schema fields.

Example
^^^^^^^

This is an example for configuring gunicorn to emit json logs.

``gunicorn.conf.py``

.. code-block:: python

    import structlog
    import structlog_extensions

    # --- Structlog logging initialisation code

    pre_chain = [
        # Add the log level and a timestamp to the event_dict if the log entry
        # is not from structlog.
        structlog.stdlib.add_log_level,
        structlog.stdlib.add_logger_name,
        structlog_extensions.processors.CombinedLogParser("gunicorn.access")
    ]

    logconfig_dict = {
                "version": 1,
                "disable_existing_loggers": False,
                "formatters": {
                    "json_formatter": {
                        "()": structlog.stdlib.ProcessorFormatter,
                        "processor": structlog.processors.JSONRenderer(),
                        "foreign_pre_chain": pre_chain,
                    }
                },
                "handlers": {
                    "error_console": {
                        "class": "logging.StreamHandler",
                        "formatter": "json_formatter",
                    },
                    "console": {
                        "class": "logging.StreamHandler",
                        "formatter": "json_formatter",
                    }
                },
            }


``NestedDictJSONRenderer``
--------------------------

This processor will convert key names using a specified separator into nested dictionaries prior to rendering the
output as JSON using the structlog JSONRenderer. This processor can for example convert Elastic Common Schema namespaced
keynames produced by the ``CombinedLogParser`` into the nested JSON format that ECS specifies. This processor must be the
final processor in a chain because it renders the output as a string instead of passing along an event dictionary.

Example
^^^^^^^

When using this logging initialisation:

        .. code-block:: python

            # --- std logging initialisation code using structlog rendering
            import structlog
            import structlog_extensions

            pre_chain = [
                # Add the log level and a timestamp to the event_dict if the log entry
                # is not from structlog.
                structlog.stdlib.add_log_level,
                structlog.stdlib.add_logger_name,
                structlog_extensions.processors.CombinedLogParser("gunicorn.access")
            ]

            logging.dict_config( {
                        "version": 1,
                        "disable_existing_loggers": False,
                        "formatters": {
                            "json_formatter": {
                                "()": structlog.stdlib.ProcessorFormatter,
                                "processor": structlog_extensions.processors.NestedDictJSONRenderer('.'),
                                "foreign_pre_chain": pre_chain,
                            }
                        },
                        "handlers": {
                            "error_console": {
                                "class": "logging.StreamHandler",
                                "formatter": "json_formatter",
                            },
                            "console": {
                                "class": "logging.StreamHandler",
                                "formatter": "json_formatter",
                            }
                        },
                    })

        These entries (produced by ``structlog_extensions.processors.CombinedLogParser``):

        .. code-block:: python

            { 'http.request.method': 'get', 'http:.request.referrer': 'http://www.example.com', 'http.version': '1.0'}`

        will be transformed into the following nested json structure:

        .. code-block:: python

            { 'http': { 'version': '1.0',
                        'request': { 'method': 'get',
                                     'referrer': 'http://www.example.com'}
                        }
            }

.. --end-usage-

