logging
logging
¶
Structured logging infrastructure for Marianne.
Provides structured JSON logging using structlog with Marianne-specific context such as job_id, sheet_num, and component names. Supports both console and file output with log rotation.
Example usage
from marianne.core.logging import get_logger, configure_logging, with_context
Configure once at startup¶
configure_logging(LogConfig(level="DEBUG", format="console"))
Get a component-specific logger¶
logger = get_logger("runner")
Log with auto-context¶
logger.info("starting_sheet", sheet_num=5)
Bind context for a scope¶
ctx_logger = logger.bind(job_id="my-job", sheet_num=1) ctx_logger.debug("executing_prompt")
Use execution context for automatic correlation¶
from marianne.core.logging import ExecutionContext, with_context
ctx = ExecutionContext(job_id="my-job", run_id="abc-123") with with_context(ctx): logger.info("sheet_started") # Automatically includes job_id, run_id
Attributes¶
Classes¶
CompressingRotatingFileHandler
¶
CompressingRotatingFileHandler(filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=False, errors=None, compress_level=9)
Bases: RotatingFileHandler
Rotating file handler that compresses old log files with gzip.
After rotation, old log files are compressed to .gz format to save disk space. For example, marianne.log.1 becomes marianne.log.1.gz.
This handler extends the standard RotatingFileHandler with: - Automatic gzip compression of rotated files - Configurable compression level (default: 9 for best compression) - Cleanup of temporary files on compression failure
Example
handler = CompressingRotatingFileHandler( "logs/marianne.log", maxBytes=50 * 1024 * 1024, # 50MB backupCount=5, compress_level=9, )
Initialize the compressing rotating file handler.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
filename
|
str | Path
|
Path to the log file. |
required |
mode
|
str
|
File mode (default 'a' for append). |
'a'
|
maxBytes
|
int
|
Maximum file size before rotation (0 = no rotation). |
0
|
backupCount
|
int
|
Number of backup files to keep. |
0
|
encoding
|
str | None
|
File encoding (default None for system default). |
None
|
delay
|
bool
|
If True, file opening is deferred until first write. |
False
|
errors
|
str | None
|
Error handling mode for encoding errors. |
None
|
compress_level
|
int
|
Gzip compression level 1-9 (default 9, best compression). |
9
|
Source code in src/marianne/core/logging.py
Functions¶
doRollover
¶
Perform log rotation with compression.
This method: 1. Closes the current log stream 2. Rotates existing .gz files (e.g., .2.gz -> .3.gz) 3. Compresses the current log file to .1.gz 4. Opens a new log file for writing
Source code in src/marianne/core/logging.py
get_log_files
¶
Get all log files managed by this handler.
Returns:
| Type | Description |
|---|---|
list[Path]
|
List of paths to all log files (current + compressed backups), |
list[Path]
|
sorted from newest to oldest. |
Source code in src/marianne/core/logging.py
ExecutionContext
dataclass
¶
ExecutionContext(job_id, run_id=(lambda: str(uuid4()))(), sheet_num=None, component='unknown', parent_run_id=None)
Immutable context for correlating log entries across an execution.
Provides tracing/correlation identifiers that are automatically included
in all log entries when set via with_context().
Attributes:
| Name | Type | Description |
|---|---|---|
job_id |
str
|
The job identifier (from config name). |
run_id |
str
|
Unique execution run ID (UUID), unique per |
sheet_num |
int | None
|
Current sheet number being processed (None if not in sheet). |
component |
str
|
Component name for the current operation (e.g., "runner", "backend"). |
parent_run_id |
str | None
|
Optional parent run ID for nested operations (e.g., sub-jobs). |
Functions¶
with_sheet
¶
Create a new context with the specified sheet number.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sheet_num
|
int
|
The sheet number to set. |
required |
Returns:
| Type | Description |
|---|---|
ExecutionContext
|
A new ExecutionContext with the sheet_num field updated. |
Source code in src/marianne/core/logging.py
with_component
¶
Create a new context with the specified component.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
component
|
str
|
The component name to set. |
required |
Returns:
| Type | Description |
|---|---|
ExecutionContext
|
A new ExecutionContext with the component field updated. |
Source code in src/marianne/core/logging.py
as_child
¶
Create a child context for nested operations.
Creates a new context where the current run_id becomes the parent_run_id, and a new run_id is generated (or uses provided child_run_id).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
child_run_id
|
str | None
|
Optional run ID for the child context. |
None
|
Returns:
| Type | Description |
|---|---|
ExecutionContext
|
A new ExecutionContext as a child of the current context. |
Source code in src/marianne/core/logging.py
to_dict
¶
Convert context to a dictionary for logging.
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
Dictionary with all context fields (excludes None values). |
Source code in src/marianne/core/logging.py
MarianneLogger
¶
Marianne-specific logger wrapper around structlog.
Provides methods for debug, info, warning, error, and critical logging with automatic inclusion of Marianne context (job_id, sheet_num, component).
The logger is bound to a component name and can have additional context bound for a specific scope (e.g., job_id, sheet_num).
Note: This class uses lazy logger initialization to ensure that loggers created at module import time still respect configuration set later via configure_logging().
Initialize a Marianne logger for a component.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
component
|
str
|
The component name (e.g., "runner", "backend", "validator"). |
required |
**initial_context
|
Any
|
Additional context to bind (e.g., job_id). |
{}
|
Source code in src/marianne/core/logging.py
Functions¶
bind
¶
Create a new logger with additional bound context.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
**context
|
Any
|
Additional context to bind (e.g., job_id, sheet_num). |
{}
|
Returns:
| Type | Description |
|---|---|
MarianneLogger
|
A new MarianneLogger with the additional context bound. |
Source code in src/marianne/core/logging.py
unbind
¶
Create a new logger with specified keys removed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
*keys
|
str
|
Keys to remove from the bound context. |
()
|
Returns:
| Type | Description |
|---|---|
MarianneLogger
|
A new MarianneLogger with the specified keys unbound. |
Source code in src/marianne/core/logging.py
debug
¶
Log a debug message.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
event
|
str
|
The event name (snake_case recommended). |
required |
**kw
|
Any
|
Additional key-value pairs to include. |
{}
|
info
¶
Log an info message.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
event
|
str
|
The event name (snake_case recommended). |
required |
**kw
|
Any
|
Additional key-value pairs to include. |
{}
|
warning
¶
Log a warning message.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
event
|
str
|
The event name (snake_case recommended). |
required |
**kw
|
Any
|
Additional key-value pairs to include. |
{}
|
error
¶
Log an error message.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
event
|
str
|
The event name (snake_case recommended). |
required |
**kw
|
Any
|
Additional key-value pairs to include. |
{}
|
critical
¶
Log a critical message.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
event
|
str
|
The event name (snake_case recommended). |
required |
**kw
|
Any
|
Additional key-value pairs to include. |
{}
|
exception
¶
Log an exception with traceback.
Should be called from within an exception handler.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
event
|
str
|
The event name (snake_case recommended). |
required |
**kw
|
Any
|
Additional key-value pairs to include. |
{}
|
Source code in src/marianne/core/logging.py
Functions¶
get_current_log_path
¶
Get the currently configured log file path.
Returns:
| Type | Description |
|---|---|
Path | None
|
The Path to the current log file, or None if file logging is not enabled. |
get_default_log_path
¶
Get the default log file path for a workspace.
The default log location is {workspace}/logs/marianne.log
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
workspace
|
Path
|
The workspace directory. |
required |
Returns:
| Type | Description |
|---|---|
Path
|
Path to the default log file location. |
Source code in src/marianne/core/logging.py
find_log_files
¶
Find all log files for a workspace.
Searches for the main log file and any rotated/compressed backups.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
workspace
|
Path
|
The workspace directory. |
required |
log_path
|
Path | None
|
Optional specific log path. If None, uses default location. |
None
|
Returns:
| Type | Description |
|---|---|
list[Path]
|
List of paths to all log files (current + compressed backups), |
list[Path]
|
sorted from newest to oldest. |
Source code in src/marianne/core/logging.py
get_current_context
¶
Get the current ExecutionContext if set.
Returns:
| Type | Description |
|---|---|
ExecutionContext | None
|
The current ExecutionContext or None if not in a context block. |
set_context
¶
Set the current ExecutionContext.
Generally prefer using with_context() for automatic cleanup.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ctx
|
ExecutionContext
|
The ExecutionContext to set as current. |
required |
Source code in src/marianne/core/logging.py
clear_context
¶
with_context
¶
Context manager that sets ExecutionContext for the duration of a block.
All log calls within the block will automatically include the context fields (job_id, run_id, sheet_num, etc.) when the _add_context processor is active.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ctx
|
ExecutionContext
|
The ExecutionContext to use for the block. |
required |
Yields:
| Type | Description |
|---|---|
ExecutionContext
|
The ExecutionContext that was set. |
Example
ctx = ExecutionContext(job_id="my-job", run_id="abc-123") with with_context(ctx): logger.info("processing") # Includes job_id, run_id automatically
Source code in src/marianne/core/logging.py
configure_logging
¶
configure_logging(level='INFO', format='console', file_path=None, max_file_size_mb=50, backup_count=5, include_timestamps=True, include_context=True, compress_logs=True)
Configure Marianne structured logging.
This should be called once at application startup before any logging occurs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
level
|
Literal['DEBUG', 'INFO', 'WARNING', 'ERROR']
|
Minimum log level to capture. |
'INFO'
|
format
|
Literal['json', 'console', 'both']
|
Output format - "json" for structured, "console" for human-readable, "both" for console to stderr and JSON to file (requires file_path). |
'console'
|
file_path
|
Path | None
|
Optional file path for log output. Required if format="both". |
None
|
max_file_size_mb
|
int
|
Maximum log file size before rotation (MB). |
50
|
backup_count
|
int
|
Number of rotated log files to keep. |
5
|
include_timestamps
|
bool
|
Whether to include ISO8601 timestamps in log entries. |
True
|
include_context
|
bool
|
Whether to include ExecutionContext fields (job_id, run_id, sheet_num) in log entries when a context is active. |
True
|
compress_logs
|
bool
|
Whether to compress rotated log files with gzip (default: True). |
True
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If format="both" but file_path is not provided. |
Source code in src/marianne/core/logging.py
735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 | |
get_logger
¶
Get a Marianne logger for a component.
The returned logger will automatically include ExecutionContext fields
(job_id, run_id, sheet_num) when logging inside a with_context() block.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
component
|
str
|
The component name (e.g., "runner", "backend", "validator"). |
required |
**initial_context
|
Any
|
Additional context to bind (e.g., job_id). |
{}
|
Returns:
| Type | Description |
|---|---|
MarianneLogger
|
A MarianneLogger instance bound to the component. |
Example
logger = get_logger("runner") ctx = ExecutionContext(job_id="my-job") with with_context(ctx): logger.info("sheet_started") # Includes job_id, run_id automatically