Too many established tcp connections from the ldap proxy

Mohammed · June 1, 2018, 6:52am

Hi There,

I’m using privacyidea ldap-proxy. the proxy build too much tcp connections without closing any session(see below). Could you please help me to solve this issue?

tcp 0 0 web2.com1.nl:53214 tcp 0 0 web2.com1.nl:52716 tcp 0 0 web2.com1.nl:53998 tcp 0 0 web2.com1.nl:50584 tcp 0 0 web2.com1.nl:49022 tcp 0 0 web2.com1.nl:49312 tcp 0 0 web2.com1.nl:51226 tcp 0 0 web2.com1.nl:53344 tcp 0 0 web2.com1.nl:52120 tcp 0 0 web2.com1.nl:50264 tcp 0 0 web2.com1.nl:50356 tcp 0 0 web2.com1.nl:50200 tcp 0 0 web2.com1.nl:53082 tcp 0 0 web2.com1.nl:50906 tcp 0 0 web2.com1.nl:51614 tcp 0 0 web2.com1.nl:49840 tcp 0 0 web2.com1.nl:52080 tcp 0 0 web2.com1.nl:52624 tcp 0 0 web2.com1.nl:51516 tcp 0 0 web2.com1.nl:51894 tcp 0 0 web2.com1.nl:50288 tcp 0 0 web2.com1.nl:49884 tcp 0 0 web2.com1.nl:50644 172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED
172.31.0.245:ldap ESTABLISHED

tcp 0 0 172.31.51.114:53460 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50736 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53798 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:52134 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51256 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51414 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51060 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53262 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53214 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:52716 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53998 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50584 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:49022 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:49312 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51226 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53344 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:52120 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50264 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50356 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50200 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53082 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50906 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51614 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:49840 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:52080 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:52624 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51516 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51894 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50288 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:49884 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50644 172.31.0.245:389 ESTABLISHED off (0.00/0/0)

fredreichbier · June 4, 2018, 9:55am

Hi Mohammed,

interesting, thanks for the report! Some questions:

What version of the LDAP proxy are you running?
What service is running at 172.31.0.245 here? Is it the LDAP backend?
Could these TCP connections be special somehow? i.e. could it be that a user tried to bind with a malformed DN, or so?

We already had an issue for a similar problem which has been resolved partially: https://github.com/privacyidea/privacyidea-ldap-proxy/issues/9

Best Regards

Friedrich

Mohammed · June 6, 2018, 4:48pm

Hi Friedrich,

Thank you for the reply. I installed it from https://github.com/privacyidea/privacyidea-ldap-proxy. yes the backed LDAP server has the IP 172.31.0.245. I don’t know if this has to do with malformed DN. The nummbers of open session keep increasing even if there is no bind queries from the end users.

fredreichbier · June 7, 2018, 7:25am

Hi Mohammed,

thank you for the information! I cannot reproduce the issue with leftover connections in the ESTABLISHED state currently. I am wondering what could cause them, so I got some more questions:

Does the LDAP proxy log give any useful information?
What application are you using to authenticate users via the LDAP proxy? Maybe the application has some “ping” functionality which results in these leftover sockets.
On which OS is the LDAP proxy running?
What LDAP backend are you using – is it an AD?
What kind of user mapping are you using? Are you using TLS, the app cache? It would be cool if you could maybe provide the relevant parts of your proxy.ini (with DNs and passwords redacted, of course).
What versions of Twisted and ldaptor are installed? You can find out by running pip freeze | grep -i -e twisted -e ldaptor

The LDAP proxy shouldn’t keep that many sockets open, so I hope we can find out why

Thanks and Best Regards

Friedrich

Mohammed · June 11, 2018, 12:44pm

Hi Friedrich,

Thank you for the reply. I think this issue has to do with using ssl or starttls to the backend LDAP server.

Does the LDAP proxy log give any useful information?

I can’t find any errors in logs.

What application are you using to authenticate users via the LDAP proxy? Maybe the application has some “ping” functionality which results in these leftover sockets
- privacyIDEA

On which OS is the LDAP proxy running?

Centos 7 64bit

What LDAP backend are you using – is it an AD?

I’m using 389 directory server and as a normal behavior, the server keeps sessions open until the client close the session.

What kind of user mapping are you using? Are you using TLS, the app cache? It would be cool if you could maybe provide the relevant parts of your proxy.ini (with DNs and passwords redacted, of course).

here is my configuration

[privacyidea]
instance = https://127.0.0.1
verify = False

[ldap-backend]
endpoint = tls:host=172.31.0.245:port=636:trustRoots=/etc/privacyidea-ldap-proxy/pems
use-tls = False
test-connection = True

[service-account]
dn = “”
password =

[ldap-proxy]
endpoint = ssl:port=636:privateKey=/etc/privacyidea-ldap-proxy/ssl/server.key:certKey=/etc/privacyidea-ldap-proxy/ssl/server.crt
passthrough-binds = “uid=ddcadmin,dc=com1,dc=nl”
bind-service-account = false
allow-search = yes
allow-connection-reuse = yes
ignore-search-result-references = false
forward-anonymous-binds = yes

[user-mapping]
strategy = lookup
attribute = cn

[realm-mapping]
strategy = static
realm =

[bind-cache]
enabled = false
timeout = 3

[app-cache]
enabled = false

What versions of Twisted and ldaptor are installed?
Twisted-Core==12.2.0

Thanks a lot

fredreichbier · June 18, 2018, 8:58am

Hi Mohammed, thank you for the information! I’ll see if I can reproduce the behavior.

I have one remaining question:

What application are you using to authenticate users via the LDAP proxy? Maybe the application has some “ping” functionality which results in these leftover sockets

privacyIDEA

Here I was wondering about something different: What application sends LDAP BIND requests to the LDAP proxy in order to authenticate them? i.e. by which kind of applications is the LDAP proxy contacted on port 636? Personally I’ve used ownCloud with the user_ldap app which works pretty well. But I’ve come across applications which have some issues with the LDAP proxy.

Best Regards

Friedrich

Mohammed · June 18, 2018, 9:38am

Hi Friedrich,

Thank you for your reply. i use LDAP proxy to authenticate Sophos SSL VPN users, password manager (web based) and accces to portal (apache). most of them use open LDAP client for authentication. LDAP proxy listen on port 636 and I’m using ssl to encrypt the traffic between the applications and LDAP proxy.

Sophos firewall send a tcp sync to port 636 evey 30 sec i guess to check if the LDAP proxy up and running. could this be the issue? I will disable the health check on port 636 and see if it’s going to help?

Mohammed · June 18, 2018, 11:27am

Hi Friedrich,

You are wright. Tcp health check on port 636 was causing this issue! Thank you very much.

fredreichbier · June 18, 2018, 1:40pm

Hi Mohammed,

thank you for trying this, and nice to hear you got it working. I would still consider this a bug in the LDAP proxy, so I opened an issue. I assume the health check feature you mentioned would be something like the feature described here?

Best Regards

Friedrich

Mohammed · June 19, 2018, 1:21pm

Hi Friedrich,

Thank you very much. Yes i’m using tcp check.

fredreichbier · June 19, 2018, 1:55pm

Hi Mohammed,

thank you for the confirmation. We’ve pushed a fix to https://github.com/privacyidea/privacyidea-ldap-proxy which should resolve that problem – you are welcome to try it out if you want!

Mohammed · June 26, 2018, 6:48am

Hi Friedrich,

It’s working. Thank you very very much.