Centos 7 apache internal server error

Hi all!

I have a troubles with installation on centos7. I use this instruction
I get an 500 internal server error. pi-manage runserver works fine, so only apache don’t work properly.

My log from /var/log/httpd/error_log:

[Fri Jul 21 10:26:51.483776 2017] [suexec:notice] [pid 21363] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Jul 21 10:26:51.543036 2017] [auth_digest:notice] [pid 21363] AH01757: generating secret for digest authentication ...
[Fri Jul 21 10:26:51.544502 2017] [lbmethod_heartbeat:notice] [pid 21363] AH02282: No slotmem from mod_heartmonitor
[Fri Jul 21 10:26:51.552665 2017] [mpm_prefork:notice] [pid 21363] AH00163: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations
[Fri Jul 21 10:26:51.552708 2017] [core:notice] [pid 21363] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'

My current /etc/httpd/conf.d/privacyidea.conf:

TraceEnable off
ServerSignature Off
ServerTokens Prod
WSGIPythonHome /opt/privacyIDEA
WSGISocketPrefix /var/run/wsgi

ServerAdmin admin@admin
ServerName servername
RewriteEngine On
RewriteCond %{HTTPS} !=On
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]

Listen 443 https
SSLSessionCache         shmcb:/etc/privacyidea/httpd/sslcache(512000)
SSLSessionCacheTimeout  300


ServerAdmin admin@admin
ServerName servername
DocumentRoot /var/www

Require all granted
Options FollowSymLinks
AllowOverride None

# The daemon is running as user 'privacyidea'
# This user should have access to the encKey database encryption file
WSGIDaemonProcess privacyidea python-path=/etc/privacyidea:/opt/privacyIDEA/lib/python2.7/site-packages processes=1 threads=15 display-name=%{GROUP} user=privacyidea
WSGIProcessGroup privacyidea
WSGIPassAuthorization On
WSGIScriptAlias / /etc/privacyidea/privacyideaapp.wsgi
SSLEngine On
SSLProtocol All -SSLv2 -SSLv3
SSLHonorCipherOrder On
SSLCipherSuite EECDH+AES256:DHE+AES256:EECDH+AES:EDH+AES:-SHA1:EECDH+RC4:EDH+RC4:RC4-SHA:AES256-SHA:!aNULL:!eNULL:!EXP:!LOW:!MD5
#SSL_CHECK = false
#DEBUG = true
#SSLCertificateFile /etc/pki/tls/certs/privserver.crt
#SSLCertificateKeyFile /etc/pki/tls/private/privserver.key
SSLCertificateFile /etc/privacyidea/server.crt
SSLCertificateKeyFile /etc/privacyidea/server.key

ErrorLog /var/log/httpd/pi_error.log
LogLevel warn

LogFormat "%h %l %u %t %>s \"%m %U %H\" %b \"%{Referer}i\" \"%{User-agent}i\"" privacyIDEA
CustomLog /var/log/httpd/ssl_pi_access.log privacyIDEA


SSLOptions +StdEnvVars


SSLOptions +StdEnvVars

BrowserMatch ".*MSIE.*" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0

I will be glad to receive any answers!
Thanks

Best Regards,
Alexander

Hi Alexander,
Your error.log only contains notices, no errors? Please check again.

Also check with the privacyidea.conf from the howto. Yours is slightly changed.
Kind regards
Cornelius

Hello Alexander,

can you look into /var/log/privacyidea/privacyidea.log?
I remember that errors in that log might result in “500 internal server error”.

Jochen

Hi Cornelius,
Thanks for your reply!

This is my /var/log/httpd/pi_error.log:
[Fri Jul 21 18:04:54.592529 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:180] mod_wsgi (pid=2898): Target WSGI script ‘/etc/privacyidea/privacyideaapp.wsgi’ cannot be loaded as Python module.
[Fri Jul 21 18:04:54.592622 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:180] mod_wsgi (pid=2898): Exception occurred processing WSGI script ‘/etc/privacyidea/privacyideaapp.wsgi’.
[Fri Jul 21 18:04:54.592676 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:180] Traceback (most recent call last):
[Fri Jul 21 18:04:54.592730 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:180] File “/etc/privacyidea/privacyideaapp.wsgi”, line 3, in
[Fri Jul 21 18:04:54.592894 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:180] from privacyidea.app import create_app
[Fri Jul 21 18:04:54.592977 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:180] ImportError: No module named privacyidea.app
[Fri Jul 21 18:04:55.512695 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:184] mod_wsgi (pid=2898): Target WSGI script ‘/etc/privacyidea/privacyideaapp.wsgi’ cannot be loaded as Python module.
[Fri Jul 21 18:04:55.512765 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:184] mod_wsgi (pid=2898): Exception occurred processing WSGI script ‘/etc/privacyidea/privacyideaapp.wsgi’.
[Fri Jul 21 18:04:55.512848 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:184] Traceback (most recent call last):
[Fri Jul 21 18:04:55.512906 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:184] File “/etc/privacyidea/privacyideaapp.wsgi”, line 3, in
[Fri Jul 21 18:04:55.512977 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:184] from privacyidea.app import create_app
[Fri Jul 21 18:04:55.513028 2017] [:error] [pid 2898] [remote xx.xx.xx.xx:184] ImportError: No module named privacyidea.app

and /var/log/httpd/pi_access_log:
xx.xx.xx.xx - - [24/Jul/2017:14:14:28 +0300] 500 “GET / HTTP/1.1” 541 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36”
xx.xx.xx.xx - - [24/Jul/2017:14:14:28 +0300] 500 “GET /favicon.ico HTTP/1.1” 541 “https://srv-priv-t01/” "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115$

Is it deals with python environment? I try this howto on two centos servers, the results are the same. Ubuntu installation works fine.
I suspect python wsgi not starts properly. I try to change WSGIPythonHome from /opt/privacyIDEA to /opt/privacyIDEA/bin/python2.7, change owner to privacyidea user but it not helps. I think, it would be impossible to run pi-manage runserver properly if I make a mistake during installation…

I try run privacyideaapp.wsgi content via /opt/privacyIDEA/bin/python2.7 too:

[admin@servername bin]$ sudo python2.7
Python 2.7.5 (default, Nov 6 2016, 00:28:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.

import sys
sys.stdout = sys.stderr
from privacyidea.app import create_app
No handlers could be found for logger “privacyidea.lib.stats”
application = create_app(config_name=“production”, config_file=“/etc/privacyidea/pi.cfg”)
The configuration name is: production
Additional configuration can be read from the file /etc/privacyidea/pi.cfg
The config file specified in PI_LOGCONFIG does not exist.
Could not use PI_LOGCONFIG. Using PI_LOGLEVEL and PI_LOGFILE.
Using PI_LOGLEVEL 9.
Using PI_LOGFILE /var/log/privacyidea/privacyidea.log.

As I see, it is running without errors. Maybe I have some permissions problems.

Regards,
Alexander

Hi Jochen,
Thanks for your reply!

Yes, I have “500 internal server error” among my log files.
I add more information above.

Regards,
Alexander

The docs you referenced have a comment attached that suggests to disable SELINUX.
At least it’s worth a try (but I’d prefer to have SELINUX enabled).
If it is really SELINUX, https://wiki.centos.org/HowTos/SELinux section 5 might have some hints.

On Debian/Ubuntu I’d look whether the referenced file is installed or missing.

Can you provide the output of “ls -lZ /etc/privacyidea/privacyideaapp.wsgi” ?

Output is:
-rwxr-xr-x privacyidea root ? /etc/privacyidea/privacyideaapp.wsgi

/etc/selinux/config:

This file controls the state of SELinux on the system.

SELINUX= can take one of these three values:

enforcing - SELinux security policy is enforced.

permissive - SELinux prints warnings instead of enforcing.

disabled - No SELinux policy is loaded.

SELINUX=disabled

SELINUXTYPE= can take one of three two values:

targeted - Targeted processes are protected,

minimum - Modification of targeted policy. Only selected processes are protected.

mls - Multi Level Security protection.

SELINUXTYPE=targeted

Server was rebooted, so SELINUX is disabled

Regards,
Alexander

Did disabling SELinux solve your problem?

I think it results from another issue.

The error:

indicates, that somehow the privacyIDEA module can not be found in the wsgi script.

As mentioned your apache privacyidea.conf looks a bit odd to me.

For me the following four lines work perfectly fine:

WSGIDaemonProcess privacyidea processes=1 threads=15 display-name=%{GROUP} user=privacyidea
WSGIProcessGroup privacyidea
WSGIPassAuthorization On
WSGIScriptAlias / /etc/privacyidea/privacyideaapp.wsgi

So you should adapt the wsgiDaemonProcess.

Also check that there are no previous errors and the the python traceback would only be a subsequent error.

E.g. Please check your directory /var/run/wsgi

Kind regards
Cornelius

SELinux was disabled before installation… I made new server, without any customization, it was prepared only with instruction.
I edit WSGIDaemonProcess from WSGIDaemonProcess privacyidea python-path=/etc/privacyidea:/opt/privacyIDEA/lib/python2.7/site-packages processes=1 threads=15 display-name=%{GROUP} user=privacyidea to
WSGIDaemonProcess privacyidea processes=1 threads=15 display-name=%{GROUP} user=privacyidea
It is not help to start webserver.
/var/run/wsgi directory was created, the result is the same…

I faced with several strangenesses in installation process:

  1. Can I install packages via sudo or under root only?
  2. pi-manage.py is not working for me, I use sudo pi-manage.
  3. /opt/privacyIDEA/bin/privacyidea-fix-access-rights -f /etc/privacyidea/pi.cfg -u privacyidea is not working for me. I found this script in /usr/bin/ folder
  4. There are many conf-files in /etc/httpd near privacyidea.conf. Should I disable them?

Maybe I will publish video of my installation process for your best understanding

Regards,
Alexander

It is working now))
I start play with custom WSGI scripts to understand why this problem appear. They work properly, when I add raw to custom script:
from privacyidea.app import create_app
I get error.
I try run:
sudo -H -u privacyidea python
and run 4 raws from original wsgi-script, all is fine.

So, I comment the raw:
#WSGIPythonHome /opt/privacyIDEA
in /etc/httpd/conf.d/privacyidea.conf

Maybe somebody will find it helpfull

Thanks for your help!

Regards,
Alexander