Problem Upgrading Privacyidea from 2.11 to 2.19

alessandroG77 · August 29, 2017, 4:13pm

Hi, we have problem upgrading privacyidea from 2.11 to 2.19.

Here steps by official guide:
http://privacyidea.readthedocs.io/en/latest/installation/upgrade.html

source /opt/privacyidea/bin/activate
pip install --upgrade privacyidea

All workings fine and by command pip freeze
listing all packets updated.

Then i try to do:
pi-manage db stamp 4f32a4e1bf33 -d /opt/privacyidea/lib/privacyidea/migrations
pi-manage db upgrade -d /opt/privacyidea/lib/privacyidea/migrations

command: pi-manage db stamp 4f32a4e1bf33 -d /opt/privacyidea/lib/privacyidea/migrations
fault with this error:
** pi-manage db stamp 4f32a4e1bf33 -d /opt/privacyIDEA/lib/privacyidea/migrations**
No handlers could be found for logger “privacyidea.lib.stats”
/opt/privacyIDEA/lib/python2.7/site-packages/privacyidea/app.py:58: ExtDeprecationWarning: Importing flask.ext.migrate is deprecated, use flask_migrate instead.
** from flask.ext.migrate import Migrate**
/opt/privacyIDEA/lib/python2.7/site-packages/flask_sqlalchemy/init.py:839: FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future. Set it to True or False to suppress this warning.
** 'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and '**
/opt/privacyIDEA/lib/python2.7/site-packages/flask_sqlalchemy/init.py:839: FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future. Set it to True or False to suppress this warning.
** 'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and '**

** _ _______ **
** ___ () _____ _______ __/ / _ / __/ _ |**
** / _ / __/ / |/ / _ `/ / // // // // / _// __ |**
** / .// //|/_,/_/_, ///// ||**
// /__/

Running online

Command is hanged and if i try to log in web portal
i have this error:
{“jsonrpc”: “2.0”, “signature”: “20240264171684686949326086548930324128162204350172231049170109442729188126970637952069379314514693864044574103640977098091124219221810026458460086358679738369971932197975103315225450624235234674690279438491795991375581950083768450931169872732549768416058368381696546667917911899121306626432260015618593478443935998861739732962044055102062592791546632147430088482671131453094689684399375982600085601785800189040114724490284487275189514491559844806993234341184535271954821883712267962266099738798407204453023023289443906093440898638462898552126560860592644842794485993568975438875360583818200944054934378708264537423991”, “detail”: null, “version”: “privacyIDEA 2.19.1”, “result”: {“status”: false, “error”: {“message”: “(psycopg2.ProgrammingError) column policy.check_all_resolvers does not exist\nLINE 1: …id AS policy_id, policy.active AS policy_active, policy.che…\n ^\n [SQL: ‘SELECT policy.id AS policy_id, policy.active AS policy_active, policy.check_all_resolvers AS policy_check_all_resolvers, policy.name AS policy_name, policy.scope AS policy_scope, policy.action AS policy_action, policy.realm AS policy_realm, policy.adminrealm AS policy_adminrealm, policy.resolver AS policy_resolver, policy.“user” AS policy_user, policy.client AS policy_client, policy.time AS policy_time, policy.condition AS policy_condition \nFROM policy’]”, “code”: -500}}, “time”: 1504023141.201917, “id”: 1}

Can you help me?

My db is postgres and on another machine.

Thanks a lot.

cornelinux · August 30, 2017, 3:56pm

policy.check_all_resolvers does not exist.
This is an additional policy field, that was added somewhere on the LOOOOOOONG way from 2.11.
Try to update the database schema again.

Kind regards
Cornelius

alessandroG77 · August 30, 2017, 4:42pm

Sorry but i don’t understand.

I’ ve tried to update this schema more times but error it’s the same.

Running online is last message after schema upgrade and then freeze with no output!

cornelinux · August 31, 2017, 6:49am

It should not freeze. It should return the shell prompt.

You might need to check you database log, to find out, why it is hanging.

alessandroG77 · August 31, 2017, 12:04pm

Hi cornelinux,
strange because in my log there isn’t anything…
no errors logged…

cornelinux · August 31, 2017, 10:27pm

So the question is, why the connection to the remote database hangs.
You could sniff the traffic to get more informaiton on this.

Well, you could also run the schema upgrade locally on the database server.
Install the privacyidea python package and run the pi-manage db update locally…

alessandroG77 · September 1, 2017, 10:26am

Hi great cornelinux,
ok i tried now to upgrade bin of privacyidea-> all ok.
Now upgrading not freezing but i have this error on following steps:

pi-manage db stamp 4f32a4e1bf33 -d /opt/privacyIDEA/lib/privacyidea/migrations
give me this error:
No handlers could be found for logger “privacyidea.lib.stats”
/opt/privacyIDEA/lib/python2.7/site-packages/privacyidea/app.py:58: ExtDeprecationWarning: Importing flask.ext.migrate is deprecated, use flask_migrate instead.
from flask.ext.migrate import Migrate
/opt/privacyIDEA/lib/python2.7/site-packages/flask_sqlalchemy/init.py:839: FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future. Set it to True or False to suppress this warning.
'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and '
RuntimeError: module compiled against API version 0xa but this version of numpy is 0x9
/opt/privacyIDEA/lib/python2.7/site-packages/flask_sqlalchemy/init.py:839: FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future. Set it to True or False to suppress this warning.
'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and ’

AND
pi-manage db upgrade -d /opt/privacyIDEA/lib/privacyidea/migrations

No handlers could be found for logger “privacyidea.lib.stats”
/opt/privacyIDEA/lib/python2.7/site-packages/privacyidea/app.py:58: ExtDeprecationWarning: Importing flask.ext.migrate is deprecated, use flask_migrate instead.
from flask.ext.migrate import Migrate
/opt/privacyIDEA/lib/python2.7/site-packages/flask_sqlalchemy/init.py:839: FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future. Set it to True or False to suppress this warning.
'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and '
RuntimeError: module compiled against API version 0xa but this version of numpy is 0x9
/opt/privacyIDEA/lib/python2.7/site-packages/flask_sqlalchemy/init.py:839: FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future. Set it to True or False to suppress this warning.
'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and ’

         _                    _______  _______

___ () _____ _______ __/ / _ / __/ _ |
/ _ / __/ / |/ / _ `/ / // // // // / _// __ |
/ .// //|/_,/_/_, ///// ||
// /___/

Running online
Column already exists
(psycopg2.ProgrammingError) column “Type” of relation “tokeninfo” already exists
[SQL: ‘ALTER TABLE tokeninfo ADD COLUMN “Type” VARCHAR(100)’]
Traceback (most recent call last):
File “/opt/privacyIDEA/bin/pi-manage”, line 894, in
manager.run()
File “/opt/privacyIDEA/lib/python2.7/site-packages/flask_script/init.py”, line 412, in run
result = self.handle(sys.argv[0], sys.argv[1:])
File “/opt/privacyIDEA/lib/python2.7/site-packages/flask_script/init.py”, line 383, in handle
res = handle(*args, **config)
File “/opt/privacyIDEA/lib/python2.7/site-packages/flask_script/commands.py”, line 216, in call
return self.run(*args, **kwargs)
File “/opt/privacyIDEA/lib/python2.7/site-packages/flask_migrate/init.py”, line 259, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File “/opt/privacyIDEA/lib/python2.7/site-packages/alembic/command.py”, line 254, in upgrade
script.run_env()
File “/opt/privacyIDEA/lib/python2.7/site-packages/alembic/script/base.py”, line 425, in run_env
util.load_python_file(self.dir, ‘env.py’)
File “/opt/privacyIDEA/lib/python2.7/site-packages/alembic/util/pyfiles.py”, line 93, in load_python_file
module = load_module_py(module_id, path)
File “/opt/privacyIDEA/lib/python2.7/site-packages/alembic/util/compat.py”, line 75, in load_module_py
mod = imp.load_source(module_id, path, fp)
File “/opt/privacyIDEA/lib/privacyidea/migrations/env.py”, line 87, in
run_migrations_online()
File “/opt/privacyIDEA/lib/privacyidea/migrations/env.py”, line 78, in run_migrations_online
context.run_migrations()
File “”, line 8, in run_migrations
File “/opt/privacyIDEA/lib/python2.7/site-packages/alembic/runtime/environment.py”, line 836, in run_migrations
self.get_context().run_migrations(**kw)
File “/opt/privacyIDEA/lib/python2.7/site-packages/alembic/runtime/migration.py”, line 337, in run_migrations
head_maintainer.update_to_step(step)
File “/opt/privacyIDEA/lib/python2.7/site-packages/alembic/runtime/migration.py”, line 540, in update_to_step
self.update_version(from, to_)
File “/opt/privacyIDEA/lib/python2.7/site-packages/alembic/runtime/migration.py”, line 500, in update_version
== literal_column("’%s’" % from))
File “/opt/privacyIDEA/lib/python2.7/site-packages/alembic/ddl/impl.py”, line 118, in _exec
return conn.execute(construct, *multiparams, **params)
File “/opt/privacyIDEA/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 945, in execute
return meth(self, multiparams, params)
File “/opt/privacyIDEA/lib/python2.7/site-packages/sqlalchemy/sql/elements.py”, line 263, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File “/opt/privacyIDEA/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 1053, in _execute_clauseelement
compiled_sql, distilled_params
File “/opt/privacyIDEA/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 1189, in _execute_context
context)
File “/opt/privacyIDEA/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 1402, in _handle_dbapi_exception
exc_info
File “/opt/privacyIDEA/lib/python2.7/site-packages/sqlalchemy/util/compat.py”, line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File “/opt/privacyIDEA/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 1182, in _execute_context
context)
File “/opt/privacyIDEA/lib/python2.7/site-packages/sqlalchemy/engine/default.py”, line 470, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.InternalError: (psycopg2.InternalError) current transaction is aborted, commands ignored until end of transaction block
[SQL: “UPDATE alembic_version SET version_num=‘2551ee982544’ WHERE alembic_version.version_num = ‘4f32a4e1bf33’”]

Httpd restarted

but when i try to access web interface i receive this error:
{“jsonrpc”: “2.0”, “signature”: “23856886817918389368218395258113911217475597493943105684620333156915031288164571044987257682001719237494299112410883755984231853805922636423509190590341710336654564825637984744548599914263511036230780371218661561762025334557108510882879208895664736562357452201753123644518211001667219109469243891047619065441840988351586662179027258962579922320955780478831720581116679552157806898015627056711070111035890448687409103227163411146377330400918826063663477608451206889171415936938597633275964703731024139871904638055048202211538438177636147634640311215006439261155444256231233891426616274476192222337394151170909012584366”, “detail”: null, “version”: “privacyIDEA 2.19.1”, “result”: {“status”: false, “error”: {“message”: “(psycopg2.ProgrammingError) column policy.check_all_resolvers does not exist\nLINE 1: …id AS policy_id, policy.active AS policy_active, policy.che…\n ^\n [SQL: ‘SELECT policy.id AS policy_id, policy.active AS policy_active, policy.check_all_resolvers AS policy_check_all_resolvers, policy.name AS policy_name, policy.scope AS policy_scope, policy.action AS policy_action, policy.realm AS policy_realm, policy.adminrealm AS policy_adminrealm, policy.resolver AS policy_resolver, policy.“user” AS policy_user, policy.client AS policy_client, policy.time AS policy_time, policy.condition AS policy_condition \nFROM policy’]”, “code”: -500}}, “time”: 1504261103.634437, “id”: 1}

What do you think of that?

Thank you very much…