Discussion:
Bug#848137: problem with the upgrade of tango-db
(too old to reply)
PICCA Frederic-Emmanuel
2017-01-03 15:00:07 UTC
Permalink
Raw Message
Hello, I would like to discuss about this bug [1]

I tryed to reproduce the scenary of piuparts in a virtual machine (gnome-box)

installed in 3 steps:
jessie base system
mysql-server (I need a working database)
tango-db (daemon)


It works ok, I have a running tango-db daemon (ps aux | grep tango)

then

replace jessie by stretch in the sources.list

apt-get update
apt-get install mysql-server (5.5 -> 5.6)

the tango-db is still working.


Now I installed the new default-mysql-server


apt-get install default-mysql-server (5.6 -> mariadb 10.1)

the tango-db daemon is still working.

Now I upgrade tango-db

apt-get install tango-db (tango8 -> tango9)

and during this upgrade I have a problem of right that you can see in the bug report.

I would like to know how to debug this problem, I tryed to export dbc_debug=1, But I got no real information about the failing mysqldump.

I would say that I know almost nothing about sql, but I can learn.

What is strange from mmy point of view, is that I created the table and populate it only with dbconfig-common.
So I find strange that the dump can not work.

Maybe this is a problem of compatibility between mysql/mariadb, because the dabatase was first created with mysql 5.5 and the upgrade is done via mariadb.

Are you aware of problem of right due to mariadb ?


thanks for your help


Fred

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=848137


please CC me I am not on the mailing list
PICCA Frederic-Emmanuel
2017-01-03 20:30:01 UTC
Permalink
Raw Message
I am not sure that I follow what you are doing, but if you need the code
/usr/share/dbconfig-common/data/PACKAGE/upgrade-dbadmin/DBTYPE/VERSION
/usr/share/dbconfig-common/data/PACKAGE/upgrade/DBTYPE/VERSION
Hello Paul

I just try to use dbconfig-common like I did before,
(create the tango database, populate it with a few values and create some procedures.)

What is strange to me is that with dbconfig-common (jessie version), I end up with procedure owned by
root @ localhost

but when I use the dbconfig-common of stretch, the procedure are owned by tango @ localhost 5which is right)

I always used the non dbadmin script in order to configure my database.
This is why I do not understand why the mysqldump does not work anymore.
Does the dump changed between the jessie and strecth dbconfig-common ?

By change I mean.

In jessie it is run as root, but on stretch it is run as tango ?

Cheers

Fred
Paul Gevers
2017-01-03 21:40:01 UTC
Permalink
Raw Message
Hmmm,
Post by PICCA Frederic-Emmanuel
I just try to use dbconfig-common like I did before,
(create the tango database, populate it with a few values and create some procedures.)
What is strange to me is that with dbconfig-common (jessie version), I end up with procedure owned by
I always used the non dbadmin script in order to configure my database.
This is why I do not understand why the mysqldump does not work anymore.
Does the dump changed between the jessie and strecth dbconfig-common ?
By change I mean.
In jessie it is run as root, but on stretch it is run as tango ?
Let me check the (huge) delta between jessie and stretch (hopefully
tomorrow). I made major improvements in the code, but I don't
specifically remember anything related to this issue. Furthermore, I
don't rule out that this is indeed MySQL/MariaDB related. We/you could
also ask the maintainers list of those packages if the current behavior
rings a bell. Additionally, trying to do the tests as you did so far but
sticking to MySQL instead of moving over to MariaDB would be an
interesting and insightful exercise.
I am suspecting that this commit may be related to the current behavior:
https://anonscm.debian.org/cgit/collab-maint/dbconfig-common.git/commit/?id=acdb99d61abfff54630c4cfba6e4452357a83fb9

I believe I implemented there that the drop of the database is performed
with the user privileges instead of the dbadmin privileges because I
believed one should always have the rights to drop the db. Apparently I
was wrong. We may need to clone or reassign this bug to dbconfig, but
I'm not sure yet if there aren't more things, or if tango-db should work
around the issue (which may be created by buggy dbconfig-common behavior
of the past).

Out of time now.

Paul
PICCA Frederic-Emmanuel
2017-01-04 18:10:01 UTC
Permalink
Raw Message
Hello,
Post by Paul Gevers
https://anonscm.debian.org/cgit/collab-maint/dbconfig-common.git/commit/?id=acdb99d61abfff54630c4cfba6e4452357a83fb9
I believe I implemented there that the drop of the database is performed
with the user privileges instead of the dbadmin privileges because I
believed one should always have the rights to drop the db. Apparently I
was wrong. We may need to clone or reassign this bug to dbconfig, but
I'm not sure yet if there aren't more things, or if tango-db should work
around the issue (which may be created by buggy dbconfig-common behavior
of the past).
I can not give an educated guess if the current logic of dbconfig-common is good or not.
I do not have enough knowledge of SQL/MySQL/Postsgresq etc...

This problem could affect other package using dbconfig-common.

I agreed also that I must fix the wrongly created procedure/tables due to the previous dbconfig-common behaviour.

Can you help me in this proces in order to produce the right snopset to put in my package (preinst ?) script.

I need to change the owner of the procedure from

root @ localhost -> tango @ locahost

Which kind of script should I add in my debian scripts ?

thansk for your help


Frederic
PICCA Frederic-Emmanuel
2017-01-05 17:30:01 UTC
Permalink
Raw Message
Hello,

I discuss with the tango-db upstream and he found that

this one line fixed the problem, befrore doing the tango-db upgrade

UPDATE mysql.proc SET Definer='***@localhost' where Db='tango';

Ideally it should be something like

UPDATE mysql.proc SET Definer='xxx' where Db='yyy';

where xxx is the dbuser and yyy the database name.à
It is true that for now my package works only if the database name is tango.


this is a limitation but I do not want to mix this into this bug report.

so can you help me write the right snipser at the right place in the debian scripts.

Or maybe I should just put the upgrade script og tango-db 9.2.5 intot eh dbadmin part with this fix at the end in order to have something consistant forthe next upgrade (tango 10)


Thanks for your help

Cheers

Frederic
Paul Gevers
2017-01-13 21:30:02 UTC
Permalink
Raw Message
Control: tags 850190 pending

Hi Frederic-Emmanuel,
Once I fixed 850190,
Do you think that you will fix this bug before next week in order to
let me enought time to fix tango and upload it.
I really hope I can upload this weekend. I have code that I believe does
what I want. I am in the process of testing it.
I believe that ought to work, although that is
still a hack. I was thinking of doing the "DROP PROCEDURE IF EXISTS *"
calls with the administrator credentials and the rest of the upgrade
script with the proper tango credentials. But probably your solution is
easier to implement.
I am thinking about this, and I agreed thaht it would be nice to put the fic in the postinst, just before the
dbc_go whcih does the upgrade.
Can you give me an example of how to execute this DROP PROCEDURES IF EXISTS *
with the dbadmin right in this postinst
If you want to do it outside of dbconfig-common (you write "before the
dbc_go") than I am not going to advice you, because it would be too much
work to find out the proper code (and you would need to call
dbconfig-common internals or duplicate lots of code). What I meant,
instead of the mysql code that runs as user, run a script for the
upgrade (they are run with database administrator credentials) and in
that script do two things: call the DROP PROCEDURES... and then use the
user credentials to run the normal script.
I am wondering if dbconfig-common could provide something in order to
execute an sql script everywhen at the request of the maintainer,
dbc_dbuser = dbadmin
dbc_custom_stript=<path/to/the/sql/script>
dbc_go tango "runscript"
which run the script with the right database configuration extracted from the configuration phase.
Apart from this repair, do you see more use cases? The problem is that
you would need nearly all the logic that is now in dbc_go for this to
work. What I am considering is if I could guarantee the order of
script/user mysql/admin mysql (or the last two reversed). I guess that
if I would guarantee the script to always come first, it would be easier
to solve the tango-db issue at hand (which was originally created by
dbconfig-common).

Paul
Debian Bug Tracking System
2017-01-13 21:30:02 UTC
Permalink
Raw Message
Post by Paul Gevers
tags 850190 pending
Bug #850190 [dbconfig-common] dbc: db-dump must use root credentials when needed
Added tag(s) pending.
--
848137: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=848137
850190: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850190
Debian Bug Tracking System
Contact ***@bugs.debian.org with problems
Debian Bug Tracking System
2017-01-13 21:30:03 UTC
Permalink
Raw Message
Post by Paul Gevers
tags 850190 pending
Bug #850190 [dbconfig-common] dbc: db-dump must use root credentials when needed
Ignoring request to alter tags of bug #850190 to the same tags previously set
--
850190: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850190
Debian Bug Tracking System
Contact ***@bugs.debian.org with problems
PICCA Frederic-Emmanuel
2017-01-14 10:20:01 UTC
Permalink
Raw Message
Hello Paul
Post by Paul Gevers
I really hope I can upload this weekend. I have code that I believe does
what I want. I am in the process of testing it.
thanks a lot.
Post by Paul Gevers
[...]
What I meant,
instead of the mysql code that runs as user, run a script for the
upgrade (they are run with database administrator credentials) and in
that script do two things: call the DROP PROCEDURES... and then use the
user credentials to run the normal script.
Apart from this repair, do you see more use cases? The problem is that
you would need nearly all the logic that is now in dbc_go for this to
work. What I am considering is if I could guarantee the order of
script/user mysql/admin mysql (or the last two reversed). I guess that
if I would guarantee the script to always come first, it would be easier
to solve the tango-db issue at hand (which was originally created by
 dbconfig-common).
ok, so If I understand correctl.

In my tango-db postinst, I will add a script

/usr/share/dbconfig-common/scripts/PACKAGE/upgrade/DBTYPE/VERSION

which does the fix (drop all the procedures)

then

dbconfig-common will run my normal

/usr/share/dbconfig-common/data/PACKAGE/upgrade/DBTYPE/VERSION

script.

right ?

I have more than one version of 'normal' upgrade scripts,

7.2.6
8.0.5
8.2.0
9.2.5

so I need to add the same amount of fix scripts in order to be sure that the database is fixed from the first upgrade.
7.2.6 -> 8.0.5 etc...

I just need to be sure that the database dump is done after the fix except if you run the dump at dbadmin, but In that case I would have a dump with the non fixed database.


Cheers


Frederic



Paul
Paul Gevers
2017-01-15 21:00:02 UTC
Permalink
Raw Message
Hi Picca,
Post by PICCA Frederic-Emmanuel
Hello Paul
Post by Paul Gevers
I really hope I can upload this weekend. I have code that I believe does
what I want. I am in the process of testing it.
thanks a lot.
I'll upload in the next hour.
Post by PICCA Frederic-Emmanuel
Post by Paul Gevers
What I meant,
instead of the mysql code that runs as user, run a script for the
upgrade (they are run with database administrator credentials) and in
that script do two things: call the DROP PROCEDURES... and then use the
user credentials to run the normal script.
Apart from this repair, do you see more use cases? The problem is that
you would need nearly all the logic that is now in dbc_go for this to
work. What I am considering is if I could guarantee the order of
script/user mysql/admin mysql (or the last two reversed). I guess that
if I would guarantee the script to always come first, it would be easier
to solve the tango-db issue at hand (which was originally created by
dbconfig-common).
ok, so If I understand correctl.
In my tango-db postinst, I will add a script
/usr/share/dbconfig-common/scripts/PACKAGE/upgrade/DBTYPE/VERSION
which does the fix (drop all the procedures)
That is the idea.
Post by PICCA Frederic-Emmanuel
then
dbconfig-common will run my normal
/usr/share/dbconfig-common/data/PACKAGE/upgrade/DBTYPE/VERSION
script.
right ?
Officially, no, because the documentation says: "If files exist in both
data and scripts, they will both be executed in an unspecified order."
However, the current behavior of dbconfig-common is to first run the
script and then run the admin code and then run the user code. So you
should be fine (but please test) and I'll make sure this behavior
doesn't change in stretch.
Post by PICCA Frederic-Emmanuel
I have more than one version of 'normal' upgrade scripts,
7.2.6
8.0.5
8.2.0
9.2.5
so I need to add the same amount of fix scripts in order to be sure
that the database is fixed from the first upgrade.
7.2.6 -> 8.0.5 etc...
I hadn't realized this, but yes. However, as Debian doesn't support
upgrading from (stable - 2) to stable, you may consider to only add the
fix to the update from the current stable version to the first in
stretch version, i.e. (8.1.2 to) 9.1.0. I don't think it is worth the
reduction thought.
Post by PICCA Frederic-Emmanuel
I just need to be sure that the database dump is done after the fix
except if you run the dump at dbadmin, but In that case I would have
a dump with the non fixed database.
Once my fix is in (as said, I expect to upload shortly), I don't think
it is worth it to worry about the unfixed database. The database dump
will be retried with dbadmin credentials when doing it with dbuser
credentials fails.

Paul

Paul Gevers
2017-01-08 21:40:01 UTC
Permalink
Raw Message
Hi Andreas,
Hi Frederic-Emmanuel, Andreas,
@Andreas, am I correct in the assumption that jessie to stretch with
tango-db was fine until MariaDB became the default? Or was this
migration with tango-db just never tested before? Have you seen other
dbconfig-common dependent packages fail since MariaDB became default (or
maybe since dbconfig-common 2.0.5 (26-08-2016)?
I can't remember the details right now, but it is possible that this
started to fail after the mariadb switch (which is a non-trivial thing
to test with piuparts). I could try to find some older logs ...
And I don't remember any other package having a similar issue.
I needed, I could do further checks (or log lookup) next week.
It would still be nice to confirm that this issue exists (in
experimental) since 30 Mar 2015 (or dbconfig-common release 1.8.50).

Paul
Loading...