Too many established tcp connections from the ldap proxy


#1

Hi There,

I’m using privacyidea ldap-proxy. the proxy build too much tcp connections without closing any session(see below). Could you please help me to solve this issue?

tcp 0 0 web2.com1.nl:53214 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:52716 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:53998 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:50584 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:49022 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:49312 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:51226 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:53344 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:52120 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:50264 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:50356 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:50200 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:53082 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:50906 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:51614 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:49840 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:52080 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:52624 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:51516 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:51894 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:50288 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:49884 172.31.0.245:ldap ESTABLISHED
tcp 0 0 web2.com1.nl:50644 172.31.0.245:ldap ESTABLISHED

tcp 0 0 172.31.51.114:53460 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50736 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53798 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:52134 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51256 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51414 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51060 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53262 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53214 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:52716 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53998 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50584 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:49022 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:49312 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51226 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53344 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:52120 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50264 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50356 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50200 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:53082 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50906 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51614 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:49840 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:52080 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:52624 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51516 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:51894 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50288 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:49884 172.31.0.245:389 ESTABLISHED off (0.00/0/0)
tcp 0 0 172.31.51.114:50644 172.31.0.245:389 ESTABLISHED off (0.00/0/0)


#2

Hi Mohammed,

interesting, thanks for the report! Some questions:

  • What version of the LDAP proxy are you running?
  • What service is running at 172.31.0.245 here? Is it the LDAP backend?
  • Could these TCP connections be special somehow? i.e. could it be that a user tried to bind with a malformed DN, or so?

We already had an issue for a similar problem which has been resolved partially: https://github.com/privacyidea/privacyidea-ldap-proxy/issues/9

Best Regards

Friedrich


#3

Hi Friedrich,

Thank you for the reply. I installed it from https://github.com/privacyidea/privacyidea-ldap-proxy. yes the backed LDAP server has the IP 172.31.0.245. I don’t know if this has to do with malformed DN. The nummbers of open session keep increasing even if there is no bind queries from the end users.


#4

Hi Mohammed,

thank you for the information! I cannot reproduce the issue with leftover connections in the ESTABLISHED state currently. I am wondering what could cause them, so I got some more questions:

  • Does the LDAP proxy log give any useful information?
  • What application are you using to authenticate users via the LDAP proxy? Maybe the application has some “ping” functionality which results in these leftover sockets.
  • On which OS is the LDAP proxy running?
  • What LDAP backend are you using – is it an AD?
  • What kind of user mapping are you using? Are you using TLS, the app cache? It would be cool if you could maybe provide the relevant parts of your proxy.ini (with DNs and passwords redacted, of course).
  • What versions of Twisted and ldaptor are installed? You can find out by running pip freeze | grep -i -e twisted -e ldaptor

The LDAP proxy shouldn’t keep that many sockets open, so I hope we can find out why :slight_smile:

Thanks and Best Regards

Friedrich


#5

Hi Friedrich,

Thank you for the reply. I think this issue has to do with using ssl or starttls to the backend LDAP server.

Does the LDAP proxy log give any useful information?

  • I can’t find any errors in logs.

What application are you using to authenticate users via the LDAP proxy? Maybe the application has some “ping” functionality which results in these leftover sockets
- privacyIDEA

On which OS is the LDAP proxy running?

  • Centos 7 64bit

What LDAP backend are you using – is it an AD?

  • I’m using 389 directory server and as a normal behavior, the server keeps sessions open until the client close the session.

What kind of user mapping are you using? Are you using TLS, the app cache? It would be cool if you could maybe provide the relevant parts of your proxy.ini (with DNs and passwords redacted, of course).

  • here is my configuration

[privacyidea]
instance = https://127.0.0.1
verify = False

[ldap-backend]
endpoint = tls:host=172.31.0.245:port=636:trustRoots=/etc/privacyidea-ldap-proxy/pems
use-tls = False
test-connection = True

[service-account]
dn = “”
password =

[ldap-proxy]
endpoint = ssl:port=636:privateKey=/etc/privacyidea-ldap-proxy/ssl/server.key:certKey=/etc/privacyidea-ldap-proxy/ssl/server.crt
passthrough-binds = “uid=ddcadmin,dc=com1,dc=nl”
bind-service-account = false
allow-search = yes
allow-connection-reuse = yes
ignore-search-result-references = false
forward-anonymous-binds = yes

[user-mapping]
strategy = lookup
attribute = cn

[realm-mapping]
strategy = static
realm =

[bind-cache]
enabled = false
timeout = 3

[app-cache]
enabled = false

What versions of Twisted and ldaptor are installed?
Twisted-Core==12.2.0

Thanks a lot


#6

Hi Mohammed, thank you for the information! I’ll see if I can reproduce the behavior.

I have one remaining question:

What application are you using to authenticate users via the LDAP proxy? Maybe the application has some “ping” functionality which results in these leftover sockets

  • privacyIDEA

Here I was wondering about something different: What application sends LDAP BIND requests to the LDAP proxy in order to authenticate them? i.e. by which kind of applications is the LDAP proxy contacted on port 636? Personally I’ve used ownCloud with the user_ldap app which works pretty well. But I’ve come across applications which have some issues with the LDAP proxy.

Best Regards

Friedrich


#7

Hi Friedrich,

Thank you for your reply. i use LDAP proxy to authenticate Sophos SSL VPN users, password manager (web based) and accces to portal (apache). most of them use open LDAP client for authentication. LDAP proxy listen on port 636 and I’m using ssl to encrypt the traffic between the applications and LDAP proxy.

Sophos firewall send a tcp sync to port 636 evey 30 sec i guess to check if the LDAP proxy up and running. could this be the issue? I will disable the health check on port 636 and see if it’s going to help?


#8

Hi Friedrich,

You are wright. Tcp health check on port 636 was causing this issue! Thank you very much.


#9

Hi Mohammed,

thank you for trying this, and nice to hear you got it working. I would still consider this a bug in the LDAP proxy, so I opened an issue. I assume the health check feature you mentioned would be something like the feature described here?

Best Regards

Friedrich


#10

Hi Friedrich,

Thank you very much. Yes i’m using tcp check.


#11

Hi Mohammed,

thank you for the confirmation. We’ve pushed a fix to https://github.com/privacyidea/privacyidea-ldap-proxy which should resolve that problem – you are welcome to try it out if you want!


#12

Hi Friedrich,

It’s working. Thank you very very much.