Policy Condition Client Character Limit?

wwalker · October 27, 2021, 2:37pm

I have a set of rules for a specific set of devices that I’m specifying by IP. It appears that once I specify enough IPs that exceeds 256 characters, the GUI errors out with

500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

Since these IPs are non-contiguous, I can’t use CIDR-notation. I thought creating a second profile with the same settings and just the next set of IPs would fix the issue, but the IPs in the first rule then no longer match.

Is there a designed character limitation to this function? How can I go about defining some 500 device IPs for the client field that are not contiguous.

wwalker · October 27, 2021, 6:33pm

I can see how I wasn’t quite clear in my original post…it was made with a bit of haste. I have a couple policies that should only take effect when the requesting client IP matches the policy condition client.

This had worked well in testing, but during implementation, it was identified that going beyond 256 characters in this field resulted in the HTTP 500 Internal Error.

As an attempted workaround, I created a second policy mirroring the first, and added the “overflow” IPs to this rule. Unfortunately, this resulted in all the IPs in the first rule no longer working.

Is the character limit a bug, a limitation of the GUI, a database field, or some other designed limitation?

We are running the latest version of PrivacyIDEA on Ubuntu 18.04 LTS.

cornelinux · October 27, 2021, 7:56pm

Database

github.com

privacyidea/privacyidea/blob/master/privacyidea/models.py#L1469

    
      
          check_all_resolvers = db.Column(db.Boolean, default=False)
          name = db.Column(db.Unicode(64), unique=True, nullable=False)
          scope = db.Column(db.Unicode(32), nullable=False)
          action = db.Column(db.Unicode(2000), default=u"")
          realm = db.Column(db.Unicode(256), default=u"")
          adminrealm = db.Column(db.Unicode(256), default=u"")
          adminuser = db.Column(db.Unicode(256), default=u"")
          resolver = db.Column(db.Unicode(256), default=u"")
          pinode = db.Column(db.Unicode(256), default=u"")
          user = db.Column(db.Unicode(256), default=u"")
          client = db.Column(db.Unicode(256), default=u"")
          time = db.Column(db.Unicode(64), default=u"")
          # If there are multiple matching policies, choose the one
          # with the lowest priority number. We choose 1 to be the default priotity.
          priority = db.Column(db.Integer, default=1, nullable=False)
          conditions = db.relationship("PolicyCondition",
                                       lazy="joined",
                                       backref="policy",
                                       order_by="PolicyCondition.id",
                                       # With these cascade options, we ensure that whenever a Policy object is added
                                       # to a session, its conditions are also added to the session (save-update, merge).

wwalker · October 27, 2021, 7:58pm

I realize this is more of a MySQL question, but is there any disadvantage to going in and changing the size to something like 100,000, outside of the database consuming more space?

wwalker · October 27, 2021, 8:49pm

Doing the initial database research, 100,000 is both unnecessary and impossible. If someone can verify my logic/knowledge is correct here:

The database columns use the VARCHAR data type.
Each character requires 2 bytes.
MySQL has a row limit of 65,535 bytes, leaving me with approximately 32k characters.
Subtracting the characters already present in the rest of the policy, I still have over 31k characters to play with.
- This is a bit flawed and I should use the current maximums set on each field to get a better idea of how much space I really have to work with.
Assuming worst case scenario, each IP would consume 17 characters (including the , between values).
I could comfortably increase the field limit to 30,000, allowing me to have up to 1,764 IP addresses.

All that said, is there a better way to do this? Do RADIUS clients typically send other identifiable pieces of information that are device type unique?

cornelinux · October 30, 2021, 9:44am

This really sounds like a bad approach to me, managing thousands of client IPs in privacyIDEA.

You should problably somewhere map the client IPs to certain groups. I guess you only have a handful of different Groups or roles how you want to characterize the clients.

I then would add additional information to the REST request. Using the extended policy condition you can match e.g. on every header attribute.