Hi,
First of all thnaks for the great job, this software has so many features.
I’m working on deploying it, with Miniconda and Gunicorn so I’m sharing it with anyone who wants to use it or modify it though it is still a work in progress GitHub - cochennec/privacyidea-ansible-docker (BTW, is it alright I put an MIT license on it? Is it compatible with PrivacyIdea license?)
Anyway, almost everything works very fine with my two projects that do the above, with docker first, then with ansible https://docs.ansible.com/
- With Docker, everything works
- With Ansible, Everything works almost, the env creation fails (it’s a conda env, mostly like a python virtualenv) but I do it in the terminal, then the app starts, I try to log as admin, and get error 500
Authentication failed. 500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
To understand how it works, I have a systemctl service that looks like this
[Unit]
Description=privacyidea
After=network.target
[Service]
PermissionsStartOnly = true
PIDFile = /run/privacyidea-3.5/privacyidea-3.5.pid
User=privacyidea
Group=privacyidea
WorkingDirectory=/appli/privacyidea/envs/privacyidea-3.5/etc
ExecStartPre = /bin/mkdir /run/privacyidea-3.5
ExecStartPre = /bin/chown -R privacyidea:privacyidea /run/privacyidea-3.5
ExecStart=/appli/privacyidea/envs/privacyidea-3.5/bin/run.sh
TimeoutStartSec=600
TimeoutStopSec=600
ExecReload = kill -s HUP cat $(/run/privacyidea-3.5/privacyidea-3.5.pid)
ExecStop = kill -s TERM cat $(/run/privacyidea-3.5/privacyidea-3.5.pid)
ExecStopPost = /bin/rm -rf /run/privacyidea-3.5
PrivateTmp = true
[Install]
WantedBy=multi-user.target
As you can see, it calls the /appli/privacyidea/envs/privacyidea-3.5/bin/run.sh script that looks like this
/opt/miniconda/bin/conda run -n privacyidea-3.5 gunicorn -c gunicorn.py wsgi:app
Conda run means executing the gunicorn commadn in the virtualenv privacyidea-3.5.
As it seems, it uses 3 the files in the working directory /appli/privacyidea/envs/privacyidea-3.5/etc
- config.py (privacyidea config file, that corresponds to env var PRIVACYIDEACONFIGFILE)
- wsgi.py (the app definition)
- gunicorn.py (gunicorn config file)
My config file looks like this
# The realm, where users are allowed to login as administrators
SUPERUSER_REALM = ['super', 'administrators']
# Your database mysql://u:p@host/db
SQLALCHEMY_DATABASE_URI = 'postgresql+psycopg2://privacyidea:***@***:5432/privacyidea'
# This is used to encrypt the auth_token
SECRET_KEY = 't0p s3cr3t'
# This is used to encrypt the admin passwords
PI_PEPPER = 'Never know...'
# This is used to encrypt the token data and token passwords
PI_ENCFILE = '/appli/privacyidea/envs/privacyidea-3.5/lib/python3.9/site-packages/enckey'
# This is used to sign the audit log
PI_AUDIT_KEY_PRIVATE = '/appli/privacyidea/envs/privacyidea-3.5/lib/python3.9/site-packages/private.pem'
PI_AUDIT_KEY_PUBLIC = '/appli/privacyidea/envs/privacyidea-3.5/lib/python3.9/site-packages/public.pem'
# PI_AUDIT_MODULE = <python audit module>
# PI_AUDIT_SQL_URI = <special audit log DB uri>
# PI_LOGFILE = '....'
# PI_LOGLEVEL = 20
# PI_INIT_CHECK_HOOK = 'your.module.function'
# PI_CSS = '/location/of/theme.css'
# PI_UI_DEACTIVATED = True
wsgi like this
from privacyidea.app import create_app
app = create_app(config_name='production',config_file='/appli/privacyidea/envs/privacyidea-3.5/etc/config.py',silent=True)
gunicorn like this
# Sample Gunicorn configuration file.
#
# Server socket
#
# bind - The socket to bind.
#
# A string of the form: 'HOST', 'HOST:PORT', 'unix:PATH'.
# An IP is a valid HOST.
#
# backlog - The number of pending connections. This refers
# to the number of clients that can be waiting to be
# served. Exceeding this number results in the client
# getting an error when attempting to connect. It should
# only affect servers under significant load.
#
# Must be a positive integer. Generally set in the 64-2048
# range.
#
bind = '0.0.0.0:5000'
backlog = 2048
#
# Worker processes
#
# workers - The number of worker processes that this server
# should keep alive for handling requests.
#
# A positive integer generally in the 2-4 x $(NUM_CORES)
# range. You'll want to vary this a bit to find the best
# for your particular application's work load.
#
# worker_class - The type of workers to use. The default
# sync class should handle most 'normal' types of work
# loads. You'll want to read
# http://docs.gunicorn.org/en/latest/design.html#choosing-a-worker-type
# for information on when you might want to choose one
# of the other worker classes.
#
# A string referring to a Python path to a subclass of
# gunicorn.workers.base.Worker. The default provided values
# can be seen at
# http://docs.gunicorn.org/en/latest/settings.html#worker-class
#
# worker_connections - For the eventlet and gevent worker classes
# this limits the maximum number of simultaneous clients that
# a single process can handle.
#
# A positive integer generally set to around 1000.
#
# timeout - If a worker does not notify the master process in this
# number of seconds it is killed and a new worker is spawned
# to replace it.
#
# Generally set to thirty seconds. Only set this noticeably
# higher if you're sure of the repercussions for sync workers.
# For the non sync workers it just means that the worker
# process is still communicating and is not tied to the length
# of time required to handle a single request.
#
# keepalive - The number of seconds to wait for the next request
# on a Keep-Alive HTTP connection.
#
# A positive integer. Generally set in the 1-5 seconds range.
#
workers = 4
worker_class = 'sync'
worker_connections = 1000
timeout = 30
keepalive = 2
#
# spew - Install a trace function that spews every line of Python
# that is executed when running the server. This is the
# nuclear option.
#
# True or False
#
spew = False
#
# Server mechanics
#
# daemon - Detach the main Gunicorn process from the controlling
# terminal with a standard fork/fork sequence.
#
# True or False
#
# raw_env - Pass environment variables to the execution environment.
#
# pidfile - The path to a pid file to write
#
# A path string or None to not write a pid file.
#
# user - Switch worker processes to run as this user.
#
# A valid user id (as an integer) or the name of a user that
# can be retrieved with a call to pwd.getpwnam(value) or None
# to not change the worker process user.
#
# group - Switch worker process to run as this group.
#
# A valid group id (as an integer) or the name of a user that
# can be retrieved with a call to pwd.getgrnam(value) or None
# to change the worker processes group.
#
# umask - A mask for file permissions written by Gunicorn. Note that
# this affects unix socket permissions.
#
# A valid value for the os.umask(mode) call or a string
# compatible with int(value, 0) (0 means Python guesses
# the base, so values like "0", "0xFF", "0022" are valid
# for decimal, hex, and octal representations)
#
# tmp_upload_dir - A directory to store temporary request data when
# requests are read. This will most likely be disappearing soon.
#
# A path to a directory where the process owner can write. Or
# None to signal that Python should choose one on its own.
#
daemon = True
raw_env = [
'DJANGO_SECRET_KEY=something',
'SPAM=eggs',
]
pidfile = '/run/privacyidea-3.5/privacyidea-3.5.pid'
umask = 0
user = 'privacyidea'
group = 'privacyidea'
tmp_upload_dir = None
#
# Logging
#
# logfile - The path to a log file to write to.
#
# A path string. "-" means log to stdout.
#
# loglevel - The granularity of log output
#
# A string of "debug", "info", "warning", "error", "critical"
#
errorlog = '/logs/privacyidea-gunicorn-error.log'
loglevel = 'debug'
accesslog = '/logs/privacyidea-gunicorn-access.log'
access_log_format = '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"'
#
# Process naming
#
# proc_name - A base to use with setproctitle to change the way
# that Gunicorn processes are reported in the system process
# table. This affects things like 'ps' and 'top'. If you're
# going to be running more than one instance of Gunicorn you'll
# probably want to set a name to tell them apart. This requires
# that you install the setproctitle module.
#
# A string or None to choose a default of something like 'gunicorn'.
#
proc_name = None
#
# Server hooks
#
# post_fork - Called just after a worker has been forked.
#
# A callable that takes a server and worker instance
# as arguments.
#
# pre_fork - Called just prior to forking the worker subprocess.
#
# A callable that accepts the same arguments as after_fork
#
# pre_exec - Called just prior to forking off a secondary
# master process during things like config reloading.
#
# A callable that takes a server instance as the sole argument.
#
def post_fork(server, worker):
server.log.info("Worker spawned (pid: %s)", worker.pid)
def pre_fork(server, worker):
pass
def pre_exec(server):
server.log.info("Forked child, re-executing.")
def when_ready(server):
server.log.info("Server is ready. Spawning workers")
def worker_int(worker):
worker.log.info("worker received INT or QUIT signal")
## get traceback info
import threading, sys, traceback
id2name = {th.ident: th.name for th in threading.enumerate()}
code = []
for threadId, stack in sys._current_frames().items():
code.append("\n# Thread: %s(%d)" % (id2name.get(threadId,""),threadId))
for filename, lineno, name, line in traceback.extract_stack(stack):
code.append('File: "%s", line %d, in %s' % (filename,lineno, name))
if line:
code.append(" %s" % (line.strip()))
worker.log.debug("\n".join(code))
def worker_abort(worker):
worker.log.info("worker received SIGABRT signal")
I also have a post-install.sh script that configures the server before it starts, it runs with no errors
#!/bin/bash
/opt/miniconda/bin/conda run -n privacyidea-3.5 pi-manage createdb
/opt/miniconda/bin/conda run -n privacyidea-3.5 pi-manage db stamp head -d /appli/privacyidea/envs/privacyidea-3.5/lib/privacyidea/migrations
/opt/miniconda/bin/conda run -n privacyidea-3.5 pi-manage admin add admin -e *** -p ***
All the errors go to gunicorn-error.log I don’t see what’s wrong
config: gunicorn.py
bind: ['0.0.0.0:5000']
backlog: 2048
workers: 4
worker_class: sync
threads: 1
worker_connections: 1000
max_requests: 0
max_requests_jitter: 0
timeout: 30
graceful_timeout: 30
keepalive: 2
limit_request_line: 4094
limit_request_fields: 100
limit_request_field_size: 8190
reload: False
reload_engine: auto
reload_extra_files: []
spew: False
check_config: False
preload_app: False
sendfile: None
reuse_port: False
chdir: /appli/privacyidea/envs/privacyidea-3.5/etc
daemon: True
raw_env: ['DJANGO_SECRET_KEY=something', 'SPAM=eggs']
pidfile: /run/privacyidea-3.5/privacyidea-3.5.pid
worker_tmp_dir: None
user: 1002
group: 1002
umask: 0
initgroups: False
tmp_upload_dir: None
secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
forwarded_allow_ips: ['127.0.0.1']
accesslog: /logs/privacyidea-gunicorn-access.log
disable_redirect_access_to_syslog: False
access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
errorlog: /logs/privacyidea-gunicorn-error.log
loglevel: debug
capture_output: False
logger_class: gunicorn.glogging.Logger
logconfig: None
logconfig_dict: {}
syslog_addr: udp://localhost:514
syslog: False
syslog_prefix: None
syslog_facility: user
enable_stdio_inheritance: False
statsd_host: None
dogstatsd_tags:
statsd_prefix:
proc_name: None
default_proc_name: wsgi:app
pythonpath: None
paste: None
on_starting: <function OnStarting.on_starting at 0x7fb037eeaaf0>
on_reload: <function OnReload.on_reload at 0x7fb037eeac10>
when_ready: <function when_ready at 0x7fb0382eb940>
pre_fork: <function pre_fork at 0x7fb0382eb820>
post_fork: <function post_fork at 0x7fb0382eb790>
post_worker_init: <function PostWorkerInit.post_worker_init at 0x7fb037e7f0d0>
worker_int: <function worker_int at 0x7fb0382eb9d0>
worker_abort: <function worker_abort at 0x7fb0382eba60>
pre_exec: <function pre_exec at 0x7fb0382eb8b0>
pre_request: <function PreRequest.pre_request at 0x7fb037e7f550>
post_request: <function PostRequest.post_request at 0x7fb037e7f5e0>
child_exit: <function ChildExit.child_exit at 0x7fb037e7f700>
worker_exit: <function WorkerExit.worker_exit at 0x7fb037e7f820>
nworkers_changed: <function NumWorkersChanged.nworkers_changed at 0x7fb037e7f940>
on_exit: <function OnExit.on_exit at 0x7fb037e7fa60>
proxy_protocol: False
proxy_allow_ips: ['127.0.0.1']
keyfile: None
certfile: None
ssl_version: 2
cert_reqs: 0
ca_certs: None
suppress_ragged_eofs: True
do_handshake_on_connect: False
ciphers: None
raw_paste_global_conf: []
strip_header_spaces: False
[2021-01-29 23:39:31 +0100] [4582] [INFO] Starting gunicorn 20.0.4
[2021-01-29 23:39:31 +0100] [4582] [DEBUG] Arbiter booted
[2021-01-29 23:39:31 +0100] [4582] [INFO] Listening at: http://0.0.0.0:5000 (4582)
[2021-01-29 23:39:31 +0100] [4582] [INFO] Using worker: sync
[2021-01-29 23:39:31 +0100] [4582] [INFO] Server is ready. Spawning workers
[2021-01-29 23:39:31 +0100] [4583] [INFO] Booting worker with pid: 4583
[2021-01-29 23:39:31 +0100] [4583] [INFO] Worker spawned (pid: 4583)
[2021-01-29 23:39:31 +0100] [4584] [INFO] Booting worker with pid: 4584
[2021-01-29 23:39:31 +0100] [4584] [INFO] Worker spawned (pid: 4584)
[2021-01-29 23:39:31 +0100] [4585] [INFO] Booting worker with pid: 4585
[2021-01-29 23:39:31 +0100] [4585] [INFO] Worker spawned (pid: 4585)
[2021-01-29 23:39:31 +0100] [4586] [INFO] Booting worker with pid: 4586
[2021-01-29 23:39:31 +0100] [4586] [INFO] Worker spawned (pid: 4586)
[2021-01-29 23:39:31 +0100] [4582] [DEBUG] 4 workers
[2021-01-29 23:41:00 +0100] [4582] [INFO] Handling signal: term
[2021-01-29 23:41:01 +0100] [4582] [INFO] Shutting down: Master
[2021-01-29 23:42:51 +0100] [5546] [DEBUG] Current configuration:
config: gunicorn.py
bind: ['0.0.0.0:5000']
backlog: 2048
workers: 4
worker_class: sync
threads: 1
worker_connections: 1000
max_requests: 0
max_requests_jitter: 0
timeout: 30
graceful_timeout: 30
keepalive: 2
limit_request_line: 4094
limit_request_fields: 100
limit_request_field_size: 8190
reload: False
reload_engine: auto
reload_extra_files: []
spew: False
check_config: False
preload_app: False
sendfile: None
reuse_port: False
chdir: /appli/privacyidea/envs/privacyidea-3.5/etc
daemon: True
raw_env: ['DJANGO_SECRET_KEY=something', 'SPAM=eggs']
pidfile: /run/privacyidea-3.5/privacyidea-3.5.pid
worker_tmp_dir: None
user: 1002
group: 1002
umask: 0
initgroups: False
tmp_upload_dir: None
secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
forwarded_allow_ips: ['127.0.0.1']
accesslog: /logs/privacyidea-gunicorn-access.log
disable_redirect_access_to_syslog: False
access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
errorlog: /logs/privacyidea-gunicorn-error.log
loglevel: debug
capture_output: False
logger_class: gunicorn.glogging.Logger
logconfig: None
logconfig_dict: {}
syslog_addr: udp://localhost:514
syslog: False
syslog_prefix: None
syslog_facility: user
enable_stdio_inheritance: False
statsd_host: None
dogstatsd_tags:
statsd_prefix:
proc_name: None
default_proc_name: wsgi:app
pythonpath: None
paste: None
on_starting: <function OnStarting.on_starting at 0x7f877a9f0af0>
on_reload: <function OnReload.on_reload at 0x7f877a9f0c10>
when_ready: <function when_ready at 0x7f877adf2940>
pre_fork: <function pre_fork at 0x7f877adf2820>
post_fork: <function post_fork at 0x7f877adf2790>
post_worker_init: <function PostWorkerInit.post_worker_init at 0x7f877a9850d0>
worker_int: <function worker_int at 0x7f877adf29d0>
worker_abort: <function worker_abort at 0x7f877adf2a60>
pre_exec: <function pre_exec at 0x7f877adf28b0>
pre_request: <function PreRequest.pre_request at 0x7f877a985550>
post_request: <function PostRequest.post_request at 0x7f877a9855e0>
child_exit: <function ChildExit.child_exit at 0x7f877a985700>
worker_exit: <function WorkerExit.worker_exit at 0x7f877a985820>
nworkers_changed: <function NumWorkersChanged.nworkers_changed at 0x7f877a985940>
on_exit: <function OnExit.on_exit at 0x7f877a985a60>
proxy_protocol: False
proxy_allow_ips: ['127.0.0.1']
keyfile: None
certfile: None
ssl_version: 2
cert_reqs: 0
ca_certs: None
suppress_ragged_eofs: True
do_handshake_on_connect: False
ciphers: None
raw_paste_global_conf: []
strip_header_spaces: False
[2021-01-29 23:42:51 +0100] [5546] [INFO] Starting gunicorn 20.0.4
[2021-01-29 23:42:51 +0100] [5546] [DEBUG] Arbiter booted
[2021-01-29 23:42:51 +0100] [5546] [INFO] Listening at: http://0.0.0.0:5000 (5546)
[2021-01-29 23:42:51 +0100] [5546] [INFO] Using worker: sync
[2021-01-29 23:42:51 +0100] [5546] [INFO] Server is ready. Spawning workers
[2021-01-29 23:42:51 +0100] [5547] [INFO] Booting worker with pid: 5547
[2021-01-29 23:42:51 +0100] [5547] [INFO] Worker spawned (pid: 5547)
[2021-01-29 23:42:51 +0100] [5548] [INFO] Booting worker with pid: 5548
[2021-01-29 23:42:51 +0100] [5548] [INFO] Worker spawned (pid: 5548)
[2021-01-29 23:42:51 +0100] [5549] [INFO] Booting worker with pid: 5549
[2021-01-29 23:42:51 +0100] [5549] [INFO] Worker spawned (pid: 5549)
[2021-01-29 23:42:51 +0100] [5550] [INFO] Booting worker with pid: 5550
[2021-01-29 23:42:51 +0100] [5550] [INFO] Worker spawned (pid: 5550)
[2021-01-29 23:42:51 +0100] [5546] [DEBUG] 4 workers
Nothing in access.log, nothing in privacyidea.log…
Anybody has an idea what went wrong? Only warnings I got was about enckey, audit-keys that already exist so they won’t be changed. I’d like to know if there are other log files I should be looking at.
Thanks for your help.