I’m trying to get the Audit page in the GUI output to a file, but having a hard time finding the right configuration to get me that. I can only interpret a little bit of what the config file is doing so I may have bits in there that are completely unneccessary. Below is what I am currently working with, I’m getting output to the configured file but also getting stuff I don’t want and missing stuff I want. I’m looking to record client ip, endpoint accessed, username, and domain at the very least.
End game is to have this data ingested by Logstash (Using filebeat to grab the data from the file on the PI server).
[formatters]
keys=detail
[handlers]
keys=file
[formatter_detail]
class=privacyidea.lib.log.SecureFormatter
format=[%(asctime)s][%(process)d][%(thread)d][%(levelname)s][%(name)s:%(lineno)d] %(message)s
[handler_file]
# Rollover the logfile at midnight
class=logging.handlers.RotatingFileHandler
backupCount=7
maxBytes=50000000
formatter=detail
level=INFO
args=('/var/log/privacyidea/audit.log',)
[loggers]
keys=root,privacyidea
[logger_privacyidea]
handlers=file
qualname=privacyidea
level=INFO
[logger_root]
level=INFO
handlers=file
Understanding a little more and I think where I’m hung up is understanding two things:
- Formatters: What are available to use and what specifically do each of them do.
- Loggers: What are they in layman’s terms and how do I know what information they record?
I’ve pared my configuration down to below which gives me, what looks like, exactly what the audit log in the GUI provides. However, I don’t see authentication attempts. When attempting a test authenticate against RADIUS, I see the /radiusserver endpoint getting hit but it doesn’t tell me the token or username used in the authentication attempt. When I test authentication using the PrivacyIDEA-ADFS provider, I see zero entries appear in the log. At the moment, this lets me really just audit what people are doing in the WebGUI, which is great, but I’m hoping I can record authentication attempts and some details about those attempts (token/user/ip).
[formatters]
keys=detail
[formatter_detail]
class=privacyidea.lib.log.SecureFormatter
format=[%(asctime)s][%(process)d][%(thread)d][%(levelname)s][%(name)s:%(lineno)d] %(message)s
[handlers]
keys=file
[handler_file]
class=logging.handlers.RotatingFileHandler
backupCount=7
maxBytes=50000000
formatter=detail
level=INFO
args=('/var/log/privacyidea/audit.log',)
[loggers]
keys=root
[logger_root]
level=NOTSET
handlers=file
Hi,
we have been working on the logging in the current release: privacyIDEA 3.3 Released
There is also a pretty exhaustive HowTo for working with Logstash: Event-based logging with privacyIDEA and Logstash
Also in the new release is the PI_AUDIT_LOGGER_QUALNAME configuration which sets an explicit qualname for the audit log data.
If You set this to i.e. PI_AUDIT_LOGGER_QUALNAME = 'pi-audit' and use this logging configuration:
[handler_audit]
class=logging.handlers.RotatingFileHandler
backupCount=14
maxBytes=10000000
formatter=detail
level=INFO
args=('/path/to/audit.log',)
[logger_audit]
handlers=audit
qualname=pi-audit
level=INFO
You should be able to log all audit entries into the file.
1 Like
That is a decent guide but it omits the piece I’m struggling with, which is the logging.cfg file. I was actually using filebeat to grab the files and get them over to Logstash, but I like the idea of using python-logstash-async. I’ll also try out the Logstash pipeline you’ve got configured, I was working up a ridiculous pipeline that would restructure the logs into valid json so that the json filter could split the event into fields. I haven’t even got it working for all events, just ones that start with { 'success'.
I just realized you gave me the logging.cfg *facepalm
Dear @wwalker,
is there a special reason you use logging.cfg? In the article, we give an example for both a very simple logstash-async logging.cfg and logging.yaml as well as a detailed logging.yaml.
Both formats work similar but look different.
Best,
Henning
Trying to pull logs out of PrivacyIDEA and into Elasticsearch. I want to be able to see two things, user activity in the webGUI and then everytime an authentication attempt is made against PrivacyIDEA.