Privacyidea + python9 (or python 3.10) is broken on Debian bookworm

Installation
1

Hi all,

I am upgrading my machines from Debian Bullseye to Bookworm. However after the upgrade Privacyidea is broken. Since Bookworm comes with python3.11 and that is not supported yet, I compiled and install python3.9 and also tried it with python3.10.
With those I set up a venv and installed Privacyidea 3.9 in it together with the requirements.txt for 3.9 from Github. I run Privacyidea as a wsgi-script in Apache.

Unfortunately I am unable to get it working after the Bookworm upgrade (whereas it worked fine on Bullseye). The Apache error.log shows:

[Sat Oct 14 16:21:03.021062 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354] mod_wsgi (pid=11802): Failed to exec Python script file '/srv/wsgi/lan_encrypted/www/pi/privacyidea.wsgi'.
[Sat Oct 14 16:21:03.021106 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354] mod_wsgi (pid=11802): Exception occurred processing WSGI script '/srv/wsgi/lan_encrypted/www/pi/privacyidea.wsgi'.
[Sat Oct 14 16:21:03.021233 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354] Traceback (most recent call last):
[Sat Oct 14 16:21:03.021346 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]   File "/srv/wsgi/lan_encrypted/www/pi/privacyidea.wsgi", line 13, in <module>
[Sat Oct 14 16:21:03.021350 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]     from privacyidea.app import create_app
[Sat Oct 14 16:21:03.021357 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]   File "/srv/wsgi/lan_encrypted/venvs/pi/lib/python3.9/site-packages/privacyidea/app.py", line 34, in <module>
[Sat Oct 14 16:21:03.021360 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]     import privacyidea.api.before_after
[Sat Oct 14 16:21:03.021366 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]   File "/srv/wsgi/lan_encrypted/venvs/pi/lib/python3.9/site-packages/privacyidea/api/before_after.py", line 30, in <module>
[Sat Oct 14 16:21:03.021369 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]     from .lib.utils import (send_error, get_all_params)
[Sat Oct 14 16:21:03.021375 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]   File "/srv/wsgi/lan_encrypted/venvs/pi/lib/python3.9/site-packages/privacyidea/api/lib/utils.py", line 36, in <module>
[Sat Oct 14 16:21:03.021378 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]     import jwt
[Sat Oct 14 16:21:03.021383 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]   File "/srv/wsgi/lan_encrypted/venvs/pi/lib/python3.9/site-packages/jwt/__init__.py", line 1, in <module>
[Sat Oct 14 16:21:03.021387 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]     from .api_jwk import PyJWK, PyJWKSet
[Sat Oct 14 16:21:03.021392 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]   File "/srv/wsgi/lan_encrypted/venvs/pi/lib/python3.9/site-packages/jwt/api_jwk.py", line 6, in <module>
[Sat Oct 14 16:21:03.021395 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]     from .algorithms import get_default_algorithms
[Sat Oct 14 16:21:03.021401 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]   File "/srv/wsgi/lan_encrypted/venvs/pi/lib/python3.9/site-packages/jwt/algorithms.py", line 6, in <module>
[Sat Oct 14 16:21:03.021404 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]     from .utils import (
[Sat Oct 14 16:21:03.021409 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]   File "/srv/wsgi/lan_encrypted/venvs/pi/lib/python3.9/site-packages/jwt/utils.py", line 7, in <module>
[Sat Oct 14 16:21:03.021412 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]     from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurve
[Sat Oct 14 16:21:03.021436 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]   File "/srv/wsgi/lan_encrypted/venvs/pi/lib/python3.9/site-packages/cryptography/hazmat/primitives/asymmetric/ec.py", line 11, in <module>
[Sat Oct 14 16:21:03.021439 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]     from cryptography.hazmat._oid import ObjectIdentifier
[Sat Oct 14 16:21:03.021445 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]   File "/srv/wsgi/lan_encrypted/venvs/pi/lib/python3.9/site-packages/cryptography/hazmat/_oid.py", line 9, in <module>
[Sat Oct 14 16:21:03.021448 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354]     from cryptography.hazmat.bindings._rust import (
[Sat Oct 14 16:21:03.021457 2023] [wsgi:error] [pid 11802] [remote 192.168.10.2:50354] ImportError: PyO3 modules may only be initialized once per interpreter process

What can I do to make it work?

2

Hi kvv,

I suspect that it is due to the mod_wsgi. On bookworm libapache_mod_wsgi uses libpython 3.11 by default.

bookworm_info
bookworm_info1227×712 74.1 KB

But since you are using Python3.10 or Python3.9, you also need the mod_wsgi in the corresponding libpython version(3.10 or 3.9).

You can check it if you look in your system path.

find /usr/lib/apache2/modules -name "mod_wsgi*"
...
/usr/lib/apache2/modules/mod_wsgi.so-3.11
/usr/lib/apache2/modules/mod_wsgi.so

I haven’t searched the internet if there is an apache2- mod_wsgi in version 3.9 or 3.10 for debian12.

If you find one, then use that, or go to Debian11 (bullseye) for now, there is Python 3.10 used by default and there is the corresponding libapache2-mod_wsgi package on for python 3.10 until we support Python 3.11.

BR
Julio

3

Thanks Julio, I managed to make it work.

For reference, the steps are:

./configure --enable-optimizations --prefix=/opt/python/3.9.18 --with-ensurepip=install
make altinstall
  • Fix Apache mod-wsgi:
cat << EOF >  /etc/apt/sources.list.d/debian_bullseye.list
deb http://deb.debian.org/debian bullseye main
EOF
cat << EOF >  /etc/apt/preferences.d/99-apt_wsgi_version
Package: libapache2-mod-wsgi-py3
Pin: release oldstable
Pin-Priority: 1001
EOF
apt-get update
apt-get -y --allow-downgrades install libapache2-mod-wsgi-py3
# Fix the symlink to wsgi-3.9
rm /usr/lib/apache2/modules/mod_wsgi.so
ln -s /usr/lib/apache2/modules/mod_wsgi.so-3.9 /usr/lib/apache2/modules/mod_wsgi.so

echo "export PYTHONHOME=/opt/python/3.9.18" >> /etc/apache2/envvars
systemctl restart apache2
  • Finally the usual setup: create a python-3.9 venv, install privacyidea in it and configure a vhost to use it.

Do note that as a result of this all Apache wsgi applications on the host will use Python-3.9 instead of the default Python-3.11 !

– Kees.

4

This does not seem to work at all for privacyIDEA 3.9.2.

I am using Debian 11 which ships Python 3.9 by default.

It only works when I load privacyIDEA webui from 1 single network Apache has access to, in the moment I request to other side, it runs into issues.

It is not reproducible in privacyIDEA 3.8 AFAIR.

5

@EchedelleLR what kind of issues does it run into? Do You have any logs?