Thanks for the suggestion. I’m still not seeing anything that helps me figure out what the issue might be. I added a prompt option in the pam config, and can see it was configured by looking at the debug logs, but no change in behavior.
Here is what I see in my /var/log/secure log on the system I am sshing into…
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): Setting url=https://mfa-server
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): Setting sendEmptyPass=true
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): Setting pollTime=60
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): Setting prompt=did this work
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): Setting debug=true
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): Unable to open offline file. Error: 2 No such file or directory
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): Sending request to https://mfa-server with parameters:
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): pass=0 digits
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): user=doachs2
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): {"detail": {"attributes": {"hideResponseInput": true}, "client_mode": "poll", "message": "Please confirm the authentication on your mobile device!", "messages": ["Please confirm the authentication on your mobile device!"], "multi_challenge": [{"attributes": {"hideResponseInput": true}, "client_mode": "poll", "message": "Please confirm the authentication on your mobile device!", "serial": "PIPU00063E38", "transaction_id": "01907986805304389394", "type": "push"}], "serial": "PIPU00063E38", "threadid": 140419996284608, "transaction_id": "01907986805304389394", "transaction_ids": ["01907986805304389394"], "type": "push", "preferred_client_mode": "poll"}, "id": 2, "jsonrpc": "2.0", "result": {"authentication": "CHALLENGE", "status": true, "value": false}, "time": 1771967167.132051, "version": "privacyIDEA 3.12.2", "versionnumber": "3.12.2", "signature": "rsa_sha256_pss:..."}
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): Sending request to https://mfa-server/validate/polltransaction with parameters:
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): transaction_id=01907986805304389394
Feb 24 15:06:07 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): {"detail": {"challenge_status": "pending", "threadid": 140420021462720}, "id": 2, "jsonrpc": "2.0", "result": {"authentication": "REJECT", "status": true, "value": false}, "time": 1771967167.7392142, "version": "privacyIDEA 3.12.2", "versionnumber": "3.12.2", "signature": "rsa_sha256_pss:..."}
Feb 24 15:06:11 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): Sending request to https://mfa-server/validate/check with parameters:
Feb 24 15:06:11 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): pass=0 digits
Feb 24 15:06:11 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): transaction_id=01907986805304389394
Feb 24 15:06:11 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): user=doachs2
Feb 24 15:06:11 dan pam_privacyidea[981048]: pam_privacyidea(sshd:auth): {"detail": {"message": "Found matching challenge", "serial": "PIPU00063E38", "threadid": 140420021462720}, "id": 2, "jsonrpc": "2.0", "result": {"authentication": "ACCEPT", "status": true, "value": true}, "time": 1771967171.2972777, "version": "privacyIDEA 3.12.2", "versionnumber": "3.12.2", "signature": "rsa_sha256_pss:..."}
Feb 24 15:06:11 dan sshd-session[981046]: Accepted keyboard-interactive/pam for doachs2 from fe80::2ca4:44bd:73db:9543%enp3s0 port 43774 ssh2
Feb 24 15:06:11 dan sshd-session[981046]: pam_unix(sshd:session): session opened for user doachs2(uid=1000) by doachs2(uid=0)
Maybe there is something in the logs that stands out to someone with more experience?
As soon as I add a TOTP token, I get the correct prompt. The same one that I see in the debug message. However I still don’t see the prompt that was added in the pam configuration. Here are the debug logs on the system when I add the TOTP token to that user…
Feb 24 15:17:37 dan pam_privacyidea[982497]: pam_privacyidea(sshd:auth): Setting url=https://mfa-server
Feb 24 15:17:37 dan pam_privacyidea[982497]: pam_privacyidea(sshd:auth): Setting sendEmptyPass=true
Feb 24 15:17:37 dan pam_privacyidea[982497]: pam_privacyidea(sshd:auth): Setting pollTime=60
Feb 24 15:17:37 dan pam_privacyidea[982497]: pam_privacyidea(sshd:auth): Setting prompt=did this work
Feb 24 15:17:37 dan pam_privacyidea[982497]: pam_privacyidea(sshd:auth): Setting debug=true
Feb 24 15:17:37 dan pam_privacyidea[982497]: pam_privacyidea(sshd:auth): Unable to open offline file. Error: 2 No such file or directory
Feb 24 15:17:37 dan pam_privacyidea[982497]: pam_privacyidea(sshd:auth): Sending request to https://mfa-server/validate/check with parameters:
Feb 24 15:17:37 dan pam_privacyidea[982497]: pam_privacyidea(sshd:auth): pass=0 digits
Feb 24 15:17:37 dan pam_privacyidea[982497]: pam_privacyidea(sshd:auth): user=doachs2
Feb 24 15:17:38 dan pam_privacyidea[982497]: pam_privacyidea(sshd:auth): {"detail": {"attributes": {"hideResponseInput": true}, "client_mode": "poll", "message": "Please confirm the authentication on your mobile device!, please enter otp: ", "messages": ["please enter otp: ", "Please confirm the authentication on your mobile device!"], "multi_challenge": [{"client_mode": "interactive", "message": "please enter otp: ", "serial": "TOTP0001EDAD", "transaction_id": "12622449600881198319", "type": "totp"}, {"attributes": {"hideResponseInput": true}, "client_mode": "poll", "message": "Please confirm the authentication on your mobile device!", "serial": "PIPU00063E38", "transaction_id": "12622449600881198319", "type": "push"}], "serial": "PIPU00063E38", "threadid": 140420063426240, "transaction_id": "12622449600881198319", "transaction_ids": ["12622449600881198319", "12622449600881198319"], "type": "push", "preferred_client_mode": "interactive"}, "id": 2, "jsonrpc": "2.0", "result": {"authentication": "CHALLENGE", "status": true, "value": false}, "time": 1771967857.9988673, "version": "privacyIDEA 3.12.2", "versionnumber": "3.12.2", "signature": "rsa_sha256_pss:..."}
Feb 24 15:17:39 dan sshd-session[982495]: Connection closed by authenticating user doachs2 fe80::2ca4:44bd:73db:9543%enp3s0 port 42352 [preauth]
Looking for suggestions on other steps to debug this or options to try. Thanks in advance for any assistance.
–Dan