Rancher 1.6: Github user not able to login (Internal Server Error)

Written by Claudio Kuenzler - 0 comments

Published on May 31st 2018 - Listed in Docker Rancher Linux DB Database MySQL Git

---

On our Rancher 1.6 environment one particular developer was unable to login into Rancher using his Github account. Whenever he tried it, he got an error "Internal Server Error".

But all other users were able to log in, so I first suspected a problem with his Github account (e.g. OAuth disabled or similar). On the research of the problem we came across an issue in Rancher's Github repository, having similar problems:

Some of the users from the organization added in the environment can log in with no problem, but some get Internal Server Error

The issue was closed by the OP himself with the reason:

It was because of mysql being in latin and not utf8

Could this be our problem, too?

I verified and indeed the database and all its tables were set to latin1_swedish_ci (the "old default" of MySQL). The Rancher 1.6 install documentation clearly states to create the database with utf8 though:

Instead of using the internal database that comes with Rancher server, you can start Rancher server pointing to an external database. The command would be the same, but appending in additional arguments to direct how to connect to your external database.

Here is an example of a SQL command to create a database and users.

> CREATE DATABASE IF NOT EXISTS cattle COLLATE = 'utf8_general_ci' CHARACTER SET = 'utf8';
> GRANT ALL ON cattle.* TO 'cattle'@'%' IDENTIFIED BY 'cattle';
> GRANT ALL ON cattle.* TO 'cattle'@'localhost' IDENTIFIED BY 'cattle';

Why wasn't our database created with utf8? Turns out we're using AWS RDS for this database and an option to set the collation and character set is not possible:

But can the failed login really be blamed on the database's character set? I entered the Rancher container and checked the logs:

root@rancher01:/# docker exec -it 1d0ec49acc20 bash

root@1d0ec49acc20:/# cd /var/lib/cattle/logs

root@1d0ec49acc20:/var/lib/cattle/logs# cat cattle-debug.log|grep -i github
[...]
Query is: insert into `account` (`name`, `kind`, `uuid`, `state`, `created`, `data`, `external_id`, `external_id_type`, `health_state`, `version`) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?), parameters ['Pawe? Lewandowski','user','de7b9940-e699-422f-8586-5a964adec35a','requested','2018-05-30 10:10:27.511','{}','6796097','github_user','healthy','2']
[...]

Yes, there is indeed a special character "L with stroke" in the first name. This character is not part in the latin1_swedish_ci character set. The failed login can therefore really be blamed on the wrong database/table character sets.

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.