Import Token: error after import CSV

Question
1

Hi,

I’ve initialized a new yubikey with:
privacyidea token yubikey_mass_enroll --filename secrets.csv

The resulting CSV file was imported into privacyidea 2.7, I’ve got the
message that one token has been imported. Now I switch into the token
list and get the following message:
‘NoneType’ object has no attribute 'name’
The new token is not shown in the token list (but the existing token are
still there).

I’ve looked into the database tables and find:,----
| select * from token;
| …
| | 31 | imported | UBOM03638840_1 | hotp …
`----

,----
| mysql> select * from tokenrealm;
| …
| | 39 | 31 | NULL |
`----

mysql> select * from tokeninfo where token_id = 31;
±—±--------±------±-----±------------±---------+
| id | Key | Value | Type | Description | token_id |
±—±--------±------±-----±------------±---------+
| 72 | hashlib | sha1 | NULL | NULL | 31 |
±—±--------±------±-----±------------±---------+

In /var/log/privacyidea/privacyidea.log:

[2015-10-10 14:45:41,175][1791][139759296419584][ERROR][privacyidea.app:1423] Exception on /token/ [GET]
Traceback (most recent call last):
File “/usr/lib/python2.7/dist-packages/flask/app.py”, line 1817, in wsgi_app
response = self.full_dispatch_request()
File “/usr/lib/python2.7/dist-packages/flask/app.py”, line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File “/usr/lib/python2.7/dist-packages/flask/app.py”, line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File “/usr/lib/python2.7/dist-packages/flask/app.py”, line 1475, in full_dispatch_request
rv = self.dispatch_request()
File “/usr/lib/python2.7/dist-packages/flask/app.py”, line 1461, in dispatch_request
return self.view_functionsrule.endpoint
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/log.py”, line 125, in log_wrapper
f_result = func(*args, **kwds)
File “/usr/lib/python2.7/dist-packages/privacyidea/api/token.py”, line 334, in list_api
description=description)
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/log.py”, line 125, in log_wrapper
f_result = func(*args, **kwds)
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/token.py”, line 354, in get_tokens_paginate
token_dict = tokenobject.get_as_dict()
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/tokenclass.py”, line 1284, in get_as_dict
token_dict = self.token.get()
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/log.py”, line 125, in log_wrapper
f_result = func(*args, **kwds)
File “/usr/lib/python2.7/dist-packages/privacyidea/models.py”, line 408, in get
return self.get_vars(save=save)
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/log.py”, line 125, in log_wrapper
f_result = func(*args, **kwds)
File “/usr/lib/python2.7/dist-packages/privacyidea/models.py”, line 442, in get_vars
realm_list.append(realm_entry.realm.name)
AttributeError: ‘NoneType’ object has no attribute ‘name’

Any ideas?

Jochen

–
The only problem with troubleshooting is that the trouble shoots back.

2

Hi Jochen,

I can not reproduce this.

Initially you sent this DB entries:

    I've looked into the database tables and find:Am Samstag, den 10.10.2015, 16:52 +0200 schrieb Jochen Hein:

Hi Cornelius,

Cornelius Kölbel <@cornelinux> writes:

it looks like you selected no realm when you imported the token.

When importing the token my default realm is already selected in the list.

Jochen

–
The only problem with troubleshooting is that the trouble shoots back.

    ,----
    | select * from token;
    | ...
    | | 31 | imported                  | UBOM03638840_1 | hotp ...
    `----
    
    ,----
    | mysql> select * from tokenrealm;
    | ...
    | | 39 |       31 |     NULL |
    `----

The tokenrealm table has an invalid entry for the tokenrealm of your
imported token.
I can reproduce this, when no token is selected.
Please check again.

The first time you enter the upload dialog, there might be no realm
selected. If use the select-box, you can not unselect a realm.

Would you mind to verify this again?

On the other hand, I will also take a look to improve the code, if no
uploading-realm is selected.

Kind regards
Cornelius

–
Cornelius Kölbel
@cornelinux
+49 151 2960 1417

NetKnights GmbH
http://www.netknights.it
Landgraf-Karl-Str. 19, 34131 Kassel, Germany
Tel: +49 561 3166797, Fax: +49 561 3166798

Amtsgericht Kassel, HRB 16405
Geschäftsführer: Cornelius Kölbel

signature.asc (836 Bytes)

3

Hi Cornelius,

When importing the token my default realm is already selected in the list.

I can not reproduce this.

I can :slight_smile:

Initially you sent this DB entries:

    I've looked into the database tables and find:
    ,----
    | select * from token;
    | ...
    | | 31 | imported                  | UBOM03638840_1 | hotp ...
    `----
    
    ,----
    | mysql> select * from tokenrealm;
    | ...
    | | 39 |       31 |     NULL |
    `----

The tokenrealm table has an invalid entry for the tokenrealm of your
imported token.

Yes, I agree. Setting the realm on DB level gives me a token.

I can reproduce this, when no token is selected.
Please check again.

The first time you enter the upload dialog, there might be no realm
selected. If use the select-box, you can not unselect a realm.

Would you mind to verify this again?

I did two things (after removing the token :-):

  1. select “import token”, “OATH CSV”, “jochen.org” is already selected,
    but I select another realm an reselect “jochen.org”, choose the csv file
    and import → that works.

  2. select “import token”, “OATH CSV”, “jochen.org” is already selected,
    choose the csv file and import → that doesn’t work for me.

There seems to be a problem if the realm is already selected…

Jochen–
The only problem with troubleshooting is that the trouble shoots back.

4

Who wants to setup excessive UI tests? :wink:

I fixed this behaviour:

Thanks a lot and
Kind regards
CorneliusAm Samstag, den 10.10.2015, 17:30 +0200 schrieb Jochen Hein:

Hi Cornelius,

When importing the token my default realm is already selected in the list.

I can not reproduce this.

I can :slight_smile:

Initially you sent this DB entries:

    I've looked into the database tables and find:
    ,----
    | select * from token;
    | ...
    | | 31 | imported                  | UBOM03638840_1 | hotp ...
    `----
    
    ,----
    | mysql> select * from tokenrealm;
    | ...
    | | 39 |       31 |     NULL |
    `----

The tokenrealm table has an invalid entry for the tokenrealm of your
imported token.

Yes, I agree. Setting the realm on DB level gives me a token.

I can reproduce this, when no token is selected.
Please check again.

The first time you enter the upload dialog, there might be no realm
selected. If use the select-box, you can not unselect a realm.

Would you mind to verify this again?

I did two things (after removing the token :-):

  1. select “import token”, “OATH CSV”, “jochen.org” is already selected,
    but I select another realm an reselect “jochen.org”, choose the csv file
    and import → that works.

  2. select “import token”, “OATH CSV”, “jochen.org” is already selected,
    choose the csv file and import → that doesn’t work for me.

There seems to be a problem if the realm is already selected…

Jochen

–
The only problem with troubleshooting is that the trouble shoots back.

–
Cornelius Kölbel
@cornelinux
+49 151 2960 1417

NetKnights GmbH
http://www.netknights.it
Landgraf-Karl-Str. 19, 34131 Kassel, Germany
Tel: +49 561 3166797, Fax: +49 561 3166798

Amtsgericht Kassel, HRB 16405
Geschäftsführer: Cornelius Kölbel

signature.asc (836 Bytes)

5

I did two things (after removing the token :-):

  1. select “import token”, “OATH CSV”, “jochen.org” is already selected,
    but I select another realm an reselect “jochen.org”, choose the csv file
    and import → that works.

  2. select “import token”, “OATH CSV”, “jochen.org” is already selected,
    choose the csv file and import → that doesn’t work for me.

There seems to be a problem if the realm is already selected…

Jochen

Thanks for clarifying this!

I’ll look into it!

Kind regards
CorneliusAm Samstag, den 10.10.2015, 17:30 +0200 schrieb Jochen Hein:

–
Cornelius Kölbel
@cornelinux
+49 151 2960 1417

NetKnights GmbH
http://www.netknights.it
Landgraf-Karl-Str. 19, 34131 Kassel, Germany
Tel: +49 561 3166797, Fax: +49 561 3166798

Amtsgericht Kassel, HRB 16405
Geschäftsführer: Cornelius Kölbel

signature.asc (836 Bytes)

6

Hi Cornelius,

Cornelius Kölbel cornelius.koelbel@netknights.it writes:

it looks like you selected no realm when you imported the token.

When importing the token my default realm is already selected in the list.

Jochen–
The only problem with troubleshooting is that the trouble shoots back.

7

HI Jochen,

it looks like you selected no realm when you imported the token.

This is a new feature, that you can select a realm, in which the
imported token should be located.
Obviously the code runs into a problem, when no realm is selected.
You can delete the token on the database level and reimport the CSV but
please select a realm, before importing.

Then we should open an issue for this…

Of course tokens need to be imporable without selecting a realm!

Kind regards
CorneliusAm Samstag, den 10.10.2015, 14:55 +0200 schrieb Jochen Hein:

Hi,

I’ve initialized a new yubikey with:
privacyidea token yubikey_mass_enroll --filename secrets.csv

The resulting CSV file was imported into privacyidea 2.7, I’ve got the
message that one token has been imported. Now I switch into the token
list and get the following message:
‘NoneType’ object has no attribute ‘name’
The new token is not shown in the token list (but the existing token are
still there).

I’ve looked into the database tables and find:
,----
| select * from token;
| …
| | 31 | imported | UBOM03638840_1 | hotp …
`----

,----
| mysql> select * from tokenrealm;
| …
| | 39 | 31 | NULL |
`----

mysql> select * from tokeninfo where token_id = 31;
±—±--------±------±-----±------------±---------+
| id | Key | Value | Type | Description | token_id |
±—±--------±------±-----±------------±---------+
| 72 | hashlib | sha1 | NULL | NULL | 31 |
±—±--------±------±-----±------------±---------+

In /var/log/privacyidea/privacyidea.log:

[2015-10-10 14:45:41,175][1791][139759296419584][ERROR][privacyidea.app:1423] Exception on /token/ [GET]
Traceback (most recent call last):
File “/usr/lib/python2.7/dist-packages/flask/app.py”, line 1817, in wsgi_app
response = self.full_dispatch_request()
File “/usr/lib/python2.7/dist-packages/flask/app.py”, line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File “/usr/lib/python2.7/dist-packages/flask/app.py”, line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File “/usr/lib/python2.7/dist-packages/flask/app.py”, line 1475, in full_dispatch_request
rv = self.dispatch_request()
File “/usr/lib/python2.7/dist-packages/flask/app.py”, line 1461, in dispatch_request
return self.view_functionsrule.endpoint
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/log.py”, line 125, in log_wrapper
f_result = func(*args, **kwds)
File “/usr/lib/python2.7/dist-packages/privacyidea/api/token.py”, line 334, in list_api
description=description)
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/log.py”, line 125, in log_wrapper
f_result = func(*args, **kwds)
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/token.py”, line 354, in get_tokens_paginate
token_dict = tokenobject.get_as_dict()
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/tokenclass.py”, line 1284, in get_as_dict
token_dict = self.token.get()
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/log.py”, line 125, in log_wrapper
f_result = func(*args, **kwds)
File “/usr/lib/python2.7/dist-packages/privacyidea/models.py”, line 408, in get
return self.get_vars(save=save)
File “/usr/lib/python2.7/dist-packages/privacyidea/lib/log.py”, line 125, in log_wrapper
f_result = func(*args, **kwds)
File “/usr/lib/python2.7/dist-packages/privacyidea/models.py”, line 442, in get_vars
realm_list.append(realm_entry.realm.name)
AttributeError: ‘NoneType’ object has no attribute ‘name’

Any ideas?

Jochen

–
The only problem with troubleshooting is that the trouble shoots back.

–
Cornelius Kölbel
@cornelinux
+49 151 2960 1417

NetKnights GmbH
http://www.netknights.it
Landgraf-Karl-Str. 19, 34131 Kassel, Germany
Tel: +49 561 3166797, Fax: +49 561 3166798

Amtsgericht Kassel, HRB 16405
Geschäftsführer: Cornelius Kölbel

signature.asc (836 Bytes)