Upgrading privacyIDEA 3.5 to 3.9 on RHEL8

Hi,
I have privacyIDEA 3.5 on RHEL 8, it is running with Python 3.6.8. I have installed it on virtualenv using pip. I tried to upgrade it but it failed. I used privacyidea-pip-update to upgrade, it failed with following errors:


upsieve-2.3.2.post1 urllib3-1.26.15 zipp-3.6.0
/opt/privacyidea/bin/pi-manage:65: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
  from cryptography.hazmat.backends import default_backend
Can not import grpc modules.

             _                    _______  _______
   ___  ____(_)  _____ _______ __/  _/ _ \/ __/ _ |
  / _ \/ __/ / |/ / _ `/ __/ // // // // / _// __ |
 / .__/_/ /_/|___/\_,_/\__/\_, /___/____/___/_/ |_|
/_/                       /___/
                                             v3.9.3

++ Upgrading DB schema.
/opt/privacyidea/bin/pi-manage:65: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
  from cryptography.hazmat.backends import default_backend
Can not import grpc modules.

             _                    _______  _______
   ___  ____(_)  _____ _______ __/  _/ _ \/ __/ _ |
  / _ \/ __/ / |/ / _ `/ __/ // // // // / _// __ |
 / .__/_/ /_/|___/\_,_/\__/\_, /___/____/___/_/ |_|
/_/                       /___/
                                             v3.9.3

Running online
[]


^CTraceback (most recent call last):
  File "/opt/privacyidea/bin/privacyidea-pip-update", line 89, in <module>
  File "/opt/privacyidea/bin/privacyidea-pip-update", line 83, in main
    else:
  File "/opt/privacyidea/bin/privacyidea-pip-update", line 39, in update_db_schema

  File "/usr/lib64/python3.6/subprocess.py", line 289, in call
    return p.wait(timeout=timeout)
  File "/usr/lib64/python3.6/subprocess.py", line 1477, in wait
    (pid, sts) = self._try_wait(0)
  File "/usr/lib64/python3.6/subprocess.py", line 1424, in _try_wait
    (pid, sts) = os.waitpid(self.pid, wait_flags)
KeyboardInterrupt
Traceback (most recent call last):
  File "/opt/privacyidea/lib64/python3.6/encodings/utf_16_le.py", line 15, in decode
    def decode(input, errors='strict'):
KeyboardInterrupt

The above exception was the direct cause of the following exception:

KeyboardInterrupt: decoding with 'utf-16le' codec failed (KeyboardInterrupt: )

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/privacyidea/lib/python3.6/site-packages/flask_script/commands.py", line 216, in __call__
    return self.run(*args, **kwargs)
  File "/opt/privacyidea/lib/python3.6/site-packages/flask_migrate/__init__.py", line 96, in wrapped
    f(*args, **kwargs)
  File "/opt/privacyidea/lib/python3.6/site-packages/flask_migrate/__init__.py", line 271, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/opt/privacyidea/lib/python3.6/site-packages/alembic/command.py", line 320, in upgrade
    script.run_env()
  File "/opt/privacyidea/lib/python3.6/site-packages/alembic/script/base.py", line 563, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/opt/privacyidea/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
    module = load_module_py(module_id, path)
  File "/opt/privacyidea/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/privacyidea/lib/privacyidea/migrations/env.py", line 110, in <module>
    run_migrations_online()
  File "/opt/privacyidea/lib/privacyidea/migrations/env.py", line 100, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/opt/privacyidea/lib/python3.6/site-packages/alembic/runtime/environment.py", line 851, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/opt/privacyidea/lib/python3.6/site-packages/alembic/runtime/migration.py", line 620, in run_migrations
    step.migration_fn(**kw)
  File "/opt/privacyidea/lib/privacyidea/migrations/versions/fa07bd604a75.py", line 25, in upgrade
    action="{0!s}".format(PUSH_ACTION.FIREBASE_CONFIG))
  File "/opt/privacyidea/lib/python3.6/site-packages/privacyidea/lib/log.py", line 146, in log_wrapper
    return func(*args, **kwds)
  File "/opt/privacyidea/lib/python3.6/site-packages/privacyidea/lib/policy.py", line 584, in list_policies
    reduced_policies = self.policies
  File "/opt/privacyidea/lib/python3.6/site-packages/privacyidea/lib/policy.py", line 506, in policies
    return get_config_object().policies
  File "/opt/privacyidea/lib/python3.6/site-packages/privacyidea/lib/config.py", line 349, in get_config_object
    store['config_object'] = shared_config.reload_and_clone()
  File "/opt/privacyidea/lib/python3.6/site-packages/privacyidea/lib/config.py", line 202, in reload_and_clone
    self._reload_from_db()
  File "/opt/privacyidea/lib/python3.6/site-packages/privacyidea/lib/config.py", line 137, in _reload_from_db
    for realm in Realm.query.all():
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2773, in all
    return self._iter().all()
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2919, in _iter
    execution_options={"_sa_orm_load_options": self.load_options},
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1714, in execute
    result = conn._execute_20(statement, params or {}, execution_options)
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 335, in _execute_on_connection
    self, multiparams, params, execution_options
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1582, in _execute_clauseelement
    cache_hit=cache_hit,
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1944, in _execute_context
    e, statement, parameters, cursor, context
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2128, in _handle_dbapi_exception
    util.raise_(exc_info[1], with_traceback=exc_info[2])
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1901, in _execute_context
    cursor, statement, parameters, context
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
SystemError: <class 'pyodbc.Error'> returned a result with an error set

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1057, in _rollback_impl
    self.engine.dialect.do_rollback(self.connection)
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/dialects/mssql/base.py", line 2845, in do_rollback
(privacyidea) [privacyidea@waldevddxmfa01 ~]$     super(MSDialect, self).do_rollback(dbapi_connection)
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 683, in do_rollback
    dbapi_connection.rollback()
pyodbc.OperationalError: ('08S01', '[08S01] [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure (0) (SQLEndTran)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/privacyidea/bin/pi-manage", line 1739, in <module>
    manager.run()
  File "/opt/privacyidea/lib/python3.6/site-packages/flask_script/__init__.py", line 417, in run
    result = self.handle(argv[0], argv[1:])
  File "/opt/privacyidea/lib/python3.6/site-packages/flask_script/__init__.py", line 386, in handle
    res = handle(*args, **config)
  File "/opt/privacyidea/lib/python3.6/site-packages/flask_script/commands.py", line 216, in __call__
    return self.run(*args, **kwargs)
  File "/opt/privacyidea/lib/python3.6/site-packages/flask/ctx.py", line 464, in __exit__
    self.auto_pop(exc_value)
  File "/opt/privacyidea/lib/python3.6/site-packages/flask/ctx.py", line 452, in auto_pop
    self.pop(exc)
  File "/opt/privacyidea/lib/python3.6/site-packages/flask/ctx.py", line 438, in pop
    app_ctx.pop(exc)
  File "/opt/privacyidea/lib/python3.6/site-packages/flask/ctx.py", line 238, in pop
    self.app.do_teardown_appcontext(exc)
  File "/opt/privacyidea/lib/python3.6/site-packages/flask/app.py", line 2321, in do_teardown_appcontext
    func(exc)
  File "/opt/privacyidea/lib/python3.6/site-packages/flask_sqlalchemy/__init__.py", line 899, in shutdown_session
    self.session.remove()
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 181, in remove
    self.registry().close()
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1813, in close
    self._close_impl(invalidate=False)
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1855, in _close_impl
    transaction.close(invalidate)
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 923, in close
    transaction.close()
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2416, in close
    self._do_close()
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2639, in _do_close
    self._close_impl()
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2625, in _close_impl
    self._connection_rollback_impl()
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2617, in _connection_rollback_impl
    self.connection._rollback_impl()
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1059, in _rollback_impl
    self._handle_dbapi_exception(e, None, None, None, None)
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2125, in _handle_dbapi_exception
    sqlalchemy_exception, with_traceback=exc_info[2], from_=e
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1057, in _rollback_impl
    self.engine.dialect.do_rollback(self.connection)
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/dialects/mssql/base.py", line 2845, in do_rollback
    super(MSDialect, self).do_rollback(dbapi_connection)
  File "/opt/privacyidea/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 683, in do_rollback
    dbapi_connection.rollback()
sqlalchemy.exc.OperationalError: (pyodbc.OperationalError) ('08S01', '[08S01] [Microsoft][ODBC Driver 17 for SQL Server]Communication link failure (0) (SQLEndTran)')
(Background on this error at: https://sqlalche.me/e/14/e3q8)

How can I resolve this issue?

Ctrl-C, Keyboard Interrupt?

It was stuck at
Running online
.
I tried again. First I ran “pip install --upgrade privacyidea”. It executed successfully.
Then I ran “privacyidea-schema-upgrade /opt/privacyidea/lib/privacyidea/migrations”, but it failed with following errors:

privacyidea-schema-upgrade /opt/privacyidea/lib/privacyidea/migrations
Can not import grpc modules.

         _                    _______  _______

___ () _____ _______ __/ / _ / __/ _ |
/ _ / __/ / |/ / _ `/ / // // // // / _// __ |
/ .
/
/ /
/|
/_,/_/_, ///// ||
/
/ /___/
v3.9.3

Traceback (most recent call last):
File “/opt/privacyidea/bin/pi-manage”, line 1739, in
manager.run()
File “/opt/privacyidea/lib/python3.6/site-packages/flask_script/init.py”, line 417, in run
result = self.handle(argv[0], argv[1:])
File “/opt/privacyidea/lib/python3.6/site-packages/flask_script/init.py”, line 386, in handle
res = handle(*args, **config)
File “/opt/privacyidea/lib/python3.6/site-packages/flask_script/commands.py”, line 216, in call
return self.run(*args, **kwargs)
File “/opt/privacyidea/lib/python3.6/site-packages/flask_migrate/init.py”, line 96, in wrapped
f(*args, **kwargs)
File “/opt/privacyidea/lib/python3.6/site-packages/flask_migrate/init.py”, line 369, in current
command.current(config, verbose=verbose, head_only=head_only)
File “/opt/privacyidea/lib/python3.6/site-packages/alembic/command.py”, line 517, in current
script.run_env()
File “/opt/privacyidea/lib/python3.6/site-packages/alembic/script/base.py”, line 489, in run_env
util.load_python_file(self.dir, “env.py”)
File “/opt/privacyidea/lib/python3.6/site-packages/alembic/util/pyfiles.py”, line 98, in load_python_file
module = load_module_py(module_id, path)
File “/opt/privacyidea/lib/python3.6/site-packages/alembic/util/compat.py”, line 184, in load_module_py
spec.loader.exec_module(module)
File “”, line 678, in exec_module
File “”, line 219, in _call_with_frames_removed
File “/opt/privacyidea/lib/privacyidea/migrations/env.py”, line 110, in
run_migrations_online()
File “/opt/privacyidea/lib/privacyidea/migrations/env.py”, line 100, in run_migrations_online
context.run_migrations()
File “”, line 8, in run_migrations
File “/opt/privacyidea/lib/python3.6/site-packages/alembic/runtime/environment.py”, line 846, in run_migrations
self.get_context().run_migrations(**kw)
File “/opt/privacyidea/lib/python3.6/site-packages/alembic/runtime/migration.py”, line 511, in run_migrations
for step in self._migrations_fn(heads, self):
File “/opt/privacyidea/lib/python3.6/site-packages/alembic/command.py”, line 507, in display_version
util.obfuscate_url_pw(context.connection.engine.url),
File “/opt/privacyidea/lib/python3.6/site-packages/alembic/util/messaging.py”, line 69, in obfuscate_url_pw
u.password = “XXXXX”
AttributeError: can’t set attribute
++ Stamping DB to 4f32a4e1bf33
Can not import grpc modules.

         _                    _______  _______

___ () _____ _______ __/ / _ / __/ _ |
/ _ / __/ / |/ / _ `/ / // // // // / _// __ |
/ .
/
/ /
/|
/_,/_/_, ///// ||
/
/ /___/
v3.9.3

++ Upgrading DB schema.
Can not import grpc modules.

         _                    _______  _______

___ () _____ _______ __/ / _ / __/ _ |
/ _ / __/ / |/ / _ `/ / // // // // / _// __ |
/ .
/
/ /
/|
/_,/_/_, ///// ||
/
/ /___/
v3.9.3

Running online
Column already exists
(pyodbc.ProgrammingError) (‘42S21’, “[42S21] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Column names in each table must be unique. Column name ‘Type’ in table ‘tokeninfo’ is specified more than once. (2705) (SQLExecDirectW)”)
[SQL: ALTER TABLE tokeninfo ADD [Type] NVARCHAR(100) NULL]
(Background on this error at: Error Messages — SQLAlchemy 1.4 Documentation)
(pyodbc.ProgrammingError) (‘42000’, “[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]ALTER TABLE only allows columns to be added that can contain nulls, or have a DEFAULT definition specified, or the column being added is an identity or timestamp column, or alternatively if none of the previous conditions are satisfied the table must be empty to allow addition of this column. Column ‘condition’ cannot be added to non-empty table ‘policy’ because it does not satisfy these conditions. (4901) (SQLExecDirectW)”)
[SQL: ALTER TABLE policy ADD condition INTEGER NOT NULL]
(Background on this error at: Error Messages — SQLAlchemy 1.4 Documentation)
ERROR [root] Error: Online migration expected to match one row when updating ‘2551ee982544’ to ‘2181294eed0b’ in ‘alembic_version’; 0 found