Bug 1087891 - no python3 mysql module available which works with salt.
no python3 mysql module available which works with salt.
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Containers
Current
Other Other
: P1 - Urgent : Major (vote)
: ---
Assigned To: Silvio Moioli
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-04-03 12:02 UTC by Richard Brown
Modified: 2021-06-10 12:41 UTC (History)
11 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---
jbreuer: needinfo? (mmeister)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Brown 2018-04-03 12:02:57 UTC
On current Kubic installations Velum starts, but when bootstrapping the cluster no pending nodes are ever found

Steps to Reproduce:

Install Admin Node
Wait for Velum to Start
Create account and login
Use autoyast URL in bootstrap wizard to install 3x nodes via the network
Wait for all 3 nodes to be installed
Click Next (to the Node selection screen)
Wait

Expectation:

3 Nodes should appear on the Velum node selection screen

Actual results:

Node selection screen remains empty
Comment 1 Maximilian Meister 2018-04-04 13:07:44 UTC
it seems to be "only" a connection problem between velum and the salt-api which is also shown via a flash message in /setup/discovery

because, after booting node(s) via autoyast, the keys correctly appear to the salt-master in the state unaccepted

> linux-g3qd:~ # docker exec -it $(docker ps | grep "salt-master_velum" | awk 
> '{print $1}') salt-key
> Accepted Keys:
> admin
> ca
> Denied Keys:
> Unaccepted Keys:
> 3f4ecbb2c3474654bed2bacf786e2c22
> 7dccc505586c4c2c92f2533097f6b931
> Rejected Keys:
Comment 2 Rafael Fernández López 2018-04-04 13:45:02 UTC
It looks like the mysql python library is not properly loaded by salt:

https://build.opensuse.org/package/view_file/devel:CaaSP:kubic-container/kubic-salt-master-image/kubic-salt-master-image.kiwi?expand=1

This is including `python-PyMySQL`, whereas we were including `python-MySQL-python` in IBS. Looks like salt just tries to load `MySQLdb`: https://github.com/saltstack/salt/blob/2016.11/salt/returners/mysql.py#L157-L173

Looks like PyMySQL has some compatibility layer with MySQLdb, but it doesn't export `MySQLdb` in any case so salt cannot load it.
Comment 3 Maximilian Meister 2018-04-04 14:31:05 UTC
(In reply to Rafael Fernández López from comment #2)
> It looks like the mysql python library is not properly loaded by salt:
> 
> https://build.opensuse.org/package/view_file/devel:CaaSP:kubic-container/
> kubic-salt-master-image/kubic-salt-master-image.kiwi?expand=1
> 
> This is including `python-PyMySQL`, whereas we were including
> `python-MySQL-python` in IBS. Looks like salt just tries to load `MySQLdb`:
> https://github.com/saltstack/salt/blob/2016.11/salt/returners/mysql.py#L157-
> L173
> 
> Looks like PyMySQL has some compatibility layer with MySQLdb, but it doesn't
> export `MySQLdb` in any case so salt cannot load it.

thanks for looking into it. i wonder why we use such an old and out of maintenance library, the (probably literally) last release was Jan 2014

any idea what we need to do if we'd like to port our salt master to PyMySQL? would we need to patch the salt version, or is that something we need to propose to upstream salt?
Comment 4 Thorsten Kukuk 2018-04-04 15:26:47 UTC
(In reply to Maximilian Meister from comment #3)

> any idea what we need to do if we'd like to port our salt master to PyMySQL?
> would we need to patch the salt version, or is that something we need to
> propose to upstream salt?

Let's include our salt maintainers.
Comment 5 Klaus Kämpf 2018-04-04 15:50:45 UTC
(In reply to Maximilian Meister from comment #3)
> any idea what we need to do if we'd like to port our salt master to PyMySQL?
> would we need to patch the salt version, or is that something we need to
> propose to upstream salt?

Does https://build.opensuse.org/package/show/systemsmanagement:saltstack:products:next/salt
improve the situation ?
Comment 6 Maximilian Meister 2018-04-05 06:18:56 UTC
(In reply to Klaus Kämpf from comment #5)
> (In reply to Maximilian Meister from comment #3)
> > any idea what we need to do if we'd like to port our salt master to PyMySQL?
> > would we need to patch the salt version, or is that something we need to
> > propose to upstream salt?
> 
> Does
> https://build.opensuse.org/package/show/systemsmanagement:saltstack:products:
> next/salt
> improve the situation ?

hi klaus, thanks for the hint. trying to install this version, however it is missing some dependencies:

linux-3ken:~ # transactional-update reboot pkg install salt-2018.1.99-25.1.x86_64.rpm 
Loading repository data...
Reading installed packages...
Resolving package dependencies...

Problem: nothing provides python2-salt = 2018.1.99-25.1 needed by salt-2018.1.99-25.1.x86_64
 Solution 1: do not install salt-2018.1.99-25.1.x86_64
 Solution 2: break salt-2018.1.99-25.1.x86_64 by ignoring some of its dependencies

richard, do you think this might have to do with the python version and can you give me some background about possible issues with python3 in tumbleweed (kubic)?
Comment 7 Maximilian Meister 2018-04-05 06:24:13 UTC
(In reply to Maximilian Meister from comment #6)
> (In reply to Klaus Kämpf from comment #5)
> > (In reply to Maximilian Meister from comment #3)
> > > any idea what we need to do if we'd like to port our salt master to PyMySQL?
> > > would we need to patch the salt version, or is that something we need to
> > > propose to upstream salt?
> > 
> > Does
> > https://build.opensuse.org/package/show/systemsmanagement:saltstack:products:
> > next/salt
> > improve the situation ?
> 
> hi klaus, thanks for the hint. trying to install this version, however it is
> missing some dependencies:
> 
> linux-3ken:~ # transactional-update reboot pkg install
> salt-2018.1.99-25.1.x86_64.rpm 
> Loading repository data...
> Reading installed packages...
> Resolving package dependencies...
> 
> Problem: nothing provides python2-salt = 2018.1.99-25.1 needed by
> salt-2018.1.99-25.1.x86_64
>  Solution 1: do not install salt-2018.1.99-25.1.x86_64
>  Solution 2: break salt-2018.1.99-25.1.x86_64 by ignoring some of its
> dependencies
> 
> richard, do you think this might have to do with the python version and can
> you give me some background about possible issues with python3 in tumbleweed
> (kubic)?

sorry wrong log, it's actually python3-salt:

linux-3ken:~ # transactional-update pkg install salt-2018.1.99-25.1.x86_64.rpm 
Loading repository data...
Reading installed packages...
Resolving package dependencies...

Problem: nothing provides python3-salt = 2018.1.99-25.1 needed by salt-2018.1.99-25.1.x86_64
 Solution 1: do not install salt-2018.1.99-25.1.x86_64
 Solution 2: break salt-2018.1.99-25.1.x86_64 by ignoring some of its dependencies
Comment 8 Thorsten Kukuk 2018-04-05 06:31:19 UTC
(In reply to Klaus Kämpf from comment #5)
 
> Does
> https://build.opensuse.org/package/show/systemsmanagement:saltstack:products:
> next/salt
> improve the situation ?

By diffing the package against Factory: No, it cannot imporove the situation, since there are no changes in the relevant areas at all.
Comment 9 Maximilian Meister 2018-04-05 06:49:51 UTC
from line 218 in https://build.opensuse.org/package/view_file/systemsmanagement:saltstack/salt/salt.spec?expand=1

# Suggests:     python-MySQL-python  ## Disabled for now, originally Recommended

i suspect this disable could be the reason that it's not available. i could not find any reason in the changelog why it's commented out, i'll submit a change to systemsmanagement:saltstack/salt
Comment 10 Thorsten Kukuk 2018-04-05 07:53:26 UTC
(In reply to Maximilian Meister from comment #9)
> from line 218 in
> https://build.opensuse.org/package/view_file/systemsmanagement:saltstack/
> salt/salt.spec?expand=1
> 
> # Suggests:     python-MySQL-python  ## Disabled for now, originally
> Recommended
> 
> i suspect this disable could be the reason that it's not available. i could
> not find any reason in the changelog why it's commented out, i'll submit a
> change to systemsmanagement:saltstack/salt

It's disabled because python-MySQL-python was removed from Factory, because it's outdated. So changing it to recommends will not change anything.

I also found a bug report for salt upstream about PyMySQL, but since it got ignored it was auto-closed last year after 4 years of inactivity :(
Comment 11 Maximilian Meister 2018-04-05 08:36:59 UTC
then probably the best option would be to get rid of this mysql dependency altogether. as i understood we could do this by dropping the velum pillar api along with the mysql expernal pillar
Comment 12 Maximilian Meister 2018-04-05 09:18:27 UTC
got some feedback from the team, and the consensus atm is that it would be too intrusive and too much rework to drop mysql salt interaction, so either we make salt use PyMySQL or we provide python-MySQL-python in Factory again.

maybe we could also have this package in devel:CaaSP:kubic-container and build it into the tumbleweed kubic ISO?
Comment 13 Richard Brown 2018-04-05 09:30:26 UTC
> so either we make salt use PyMySQL or we provide python-MySQL-python in Factory again

The first sounds like the better approach, I expect the team doesn't want the burden of owning python-MySQL-python

> devel:CaaSP:kubic-container

No, devel:CaaSP:kubic-container is only for container definitions, not for packages. It's a devel project for openSUSE:Factory:Containers, not openSUSE:Factory (which then both get merged into openSUSE:Tumbleweed on publishing)

If you wish to add a package like python-MySQL-python to Factory please pick a more suitable devel project
Comment 14 Thorsten Kukuk 2018-04-05 09:54:26 UTC
(In reply to Maximilian Meister from comment #12)
> so either we make salt use PyMySQL or we provide python-MySQL-python in Factory again.

We have to fix salt using PyMySQL. python-MySQL-python was dropped because it is not compatible with python3.
Comment 15 Maximilian Meister 2018-04-05 10:48:48 UTC
there's already support for a fallback import of PyMySQL in salt modules at least [0] i can try to apply the same logic for the returners

[0] https://github.com/saltstack/salt/blob/2018.3/salt/modules/mysql.py#L64
Comment 16 Maximilian Meister 2018-04-05 12:25:36 UTC
submitted https://github.com/saltstack/salt/pull/46890
patched package is https://build.opensuse.org/package/show/home:m_meister:branches:systemsmanagement:saltstack/salt
submit request https://build.opensuse.org/request/show/593780

let's see what the maintainers/upstream says. i havent tested it yet
Comment 17 Thorsten Kukuk 2018-04-06 08:45:31 UTC
(In reply to Maximilian Meister from comment #16)
> submitted https://github.com/saltstack/salt/pull/46890
> patched package is
> https://build.opensuse.org/package/show/home:m_meister:branches:
> systemsmanagement:saltstack/salt
> submit request https://build.opensuse.org/request/show/593780
> 
> let's see what the maintainers/upstream says. i havent tested it yet

I build a new kubic-salt-master-image with it, and still get the error messages:
08:42:06,273 [ERROR   ] Could not store return for event(s) - returner 'mysql.event_return' not found.
08:42:06,277 [ERROR   ] Could not store return for event(s) - returner 'mysql.event_return' not found.
Comment 18 Maximilian Meister 2018-04-06 13:20:56 UTC
(In reply to Thorsten Kukuk from comment #17)
> I build a new kubic-salt-master-image with it, and still get the error
> messages:
> 08:42:06,273 [ERROR   ] Could not store return for event(s) - returner
> 'mysql.event_return' not found.
> 08:42:06,277 [ERROR   ] Could not store return for event(s) - returner
> 'mysql.event_return' not found.

thanks for testing, have you installed PyMySQL?
Comment 19 Maximilian Meister 2018-04-06 13:49:07 UTC
Thorsten, i also found out there are some more places upstream besides returners and modules that didnt have the drop in replacement mechanism yet. e.g. salt/pillar/mysql.py or salt/auth/mysql.py

i updated the patch in https://build.opensuse.org/package/show/home:m_meister:branches:systemsmanagement:saltstack/salt
Comment 20 Thorsten Kukuk 2018-04-06 14:52:56 UTC
(In reply to Maximilian Meister from comment #18)

> thanks for testing, have you installed PyMySQL?

Argh, yes, PyMySQL was added by somebody to the kiwi file, but the wrong version is pulled in :(
Fixed.

(In reply to Maximilian Meister from comment #19)

> i updated the patch in
> https://build.opensuse.org/package/show/home:m_meister:branches:
> systemsmanagement:saltstack/salt

New images for testing are building, thanks.
Comment 21 Thorsten Kukuk 2018-04-06 15:33:42 UTC
New error message:
15:33:51,057 [ERROR   ] Exception raised when processing __virtual__ function for salt.loaded.int.returner.mysql. Module will not be loaded: name 'HAS_MYSQL' is not defined
15:33:51,081 [WARNING ] salt.loaded.int.returner.mysql.__virtual__() is wrongly returning `None`. It should either return `True`, `False` or a new name. If you're the developer of the module 'mysql', please fix this.
Comment 22 Klaus Kämpf 2018-04-06 15:50:18 UTC
(In reply to Maximilian Meister from comment #19)
> Thorsten, i also found out there are some more places upstream besides
> returners and modules that didnt have the drop in replacement mechanism yet.
> e.g. salt/pillar/mysql.py or salt/auth/mysql.py
> 
> i updated the patch in
> https://build.opensuse.org/package/show/home:m_meister:branches:
> systemsmanagement:saltstack/salt

Awaiting your pull request at https://github.com/openSUSE/salt ;-)
Comment 23 Thorsten Kukuk 2018-04-07 08:52:28 UTC
Update: I have fixed the patch so that PyMySQL will be loaded now, but of course the interface is not 100% compatible:

08:50:08,200 [ERROR   ] Could not store events - returner 'mysql.event_return' raised exception: module 'pymysql.connections' has no attribute 'OperationalError'
08:50:14,489 [CRITICAL] Specified ext_pillar interface velum is unavailable
Comment 24 Maximilian Meister 2018-04-09 06:54:40 UTC
(In reply to Klaus Kämpf from comment #22)
> (In reply to Maximilian Meister from comment #19)
> > Thorsten, i also found out there are some more places upstream besides
> > returners and modules that didnt have the drop in replacement mechanism yet.
> > e.g. salt/pillar/mysql.py or salt/auth/mysql.py
> > 
> > i updated the patch in
> > https://build.opensuse.org/package/show/home:m_meister:branches:
> > systemsmanagement:saltstack/salt
> 
> Awaiting your pull request at https://github.com/openSUSE/salt ;-)

klaus, can i also wait for https://github.com/saltstack/salt/pull/46890 to get merged and backport it to the openSUSE fork, or what's the usual workflow?


(In reply to Thorsten Kukuk from comment #23)
> Update: I have fixed the patch so that PyMySQL will be loaded now, but of
> course the interface is not 100% compatible:
> 
> 08:50:08,200 [ERROR   ] Could not store events - returner
> 'mysql.event_return' raised exception: module 'pymysql.connections' has no
> attribute 'OperationalError'
> 08:50:14,489 [CRITICAL] Specified ext_pillar interface velum is unavailable

thorsten, can you check which version of PyMySQL is installed? the OperationalError should get imported here: https://github.com/PyMySQL/PyMySQL/blob/master/pymysql/connections.py#L25

if that doesn't work maybe we need to import the err.py explicitly... but i am no python expert, to me it looks like it should be loaded already

i have also changed again the patch after some upstream feedback to fix linting issues (the undefined HAS_MYSQLDB and an import statement)
Comment 25 Thorsten Kukuk 2018-04-09 07:46:13 UTC
(In reply to Maximilian Meister from comment #24)
> thorsten, can you check which version of PyMySQL is installed? the
> OperationalError should get imported here:
> https://github.com/PyMySQL/PyMySQL/blob/master/pymysql/connections.py#L25
> 
> if that doesn't work maybe we need to import the err.py explicitly... but i
> am no python expert, to me it looks like it should be loaded already

We have 0.7.11, the newest is 0.8, and it looks like this got added to 0.8.
I will test with 0.8 now.
Comment 26 Thorsten Kukuk 2018-04-09 08:31:09 UTC
(In reply to Thorsten Kukuk from comment #25)

> I will test with 0.8 now.

PyMySQL 0.8.0 shows the same error :(
Comment 27 Klaus Kämpf 2018-04-09 09:04:09 UTC
(In reply to Maximilian Meister from comment #24)
> 
> klaus, can i also wait for https://github.com/saltstack/salt/pull/46890 to
> get merged and backport it to the openSUSE fork

This won't happen automagically ;-)

If you want it in *our* Salt, create a pull request.

>, or what's the usual
> workflow?
> 

https://github.com/openSUSE/salt/wiki


Reach out to #saltstack on irc.suse.de and ask for help !
Comment 28 Thorsten Kukuk 2018-04-10 08:52:10 UTC
Re-assigning to salt-maintainers, this is blocking Kubic and CaaSP4 development and goes behind our knowledge.
Comment 29 Bo Maryniuk 2018-04-10 09:18:56 UTC
Well, it says as it says:

15:33:51,057 [ERROR   ] Exception raised when processing __virtual__ function for salt.loaded.int.returner.mysql. Module will not be loaded: name 'HAS_MYSQL' is not defined

Which means you have somewhere HAS_MYSQL variable that you're accessing like "if HAS_MYSQL is True" or something like that, but it is not defined prior...
Comment 30 Maximilian Meister 2018-04-10 09:57:40 UTC
(In reply to Bo Maryniuk from comment #29)
> Well, it says as it says:
> 
> 15:33:51,057 [ERROR   ] Exception raised when processing __virtual__
> function for salt.loaded.int.returner.mysql. Module will not be loaded: name
> 'HAS_MYSQL' is not defined
> 
> Which means you have somewhere HAS_MYSQL variable that you're accessing like
> "if HAS_MYSQL is True" or something like that, but it is not defined prior...

this is not an issue anymore, the issue is:

> 08:50:08,200 [ERROR   ] Could not store events - returner
> 'mysql.event_return' raised exception: module 'pymysql.connections' has no
> attribute 'OperationalError'

the PR upstream [0] has also got some more reviews and changed again, but the issue should still be the same

[0] https://github.com/saltstack/salt/pull/46890
Comment 31 Bo Maryniuk 2018-04-10 10:11:57 UTC
(In reply to Maximilian Meister from comment #30)
> this is not an issue anymore, the issue is:
> 
> > 08:50:08,200 [ERROR   ] Could not store events - returner
> > 'mysql.event_return' raised exception: module 'pymysql.connections' has no
> > attribute 'OperationalError'

Sure. It is moved to pymysql.err.OperationalError class. Like we were talking before: it is not 100% compatible and some refactoring is still needed.
Comment 32 Maximilian Meister 2018-04-10 11:25:52 UTC
(In reply to Bo Maryniuk from comment #31)
> (In reply to Maximilian Meister from comment #30)
> > this is not an issue anymore, the issue is:
> > 
> > > 08:50:08,200 [ERROR   ] Could not store events - returner
> > > 'mysql.event_return' raised exception: module 'pymysql.connections' has no
> > > attribute 'OperationalError'
> 
> Sure. It is moved to pymysql.err.OperationalError class. Like we were
> talking before: it is not 100% compatible and some refactoring is still
> needed.

thanks for the hint, fixed in https://github.com/saltstack/salt/pull/46890/commits/f4c083dcb8701fc79a0ae5909987758637daed18
Comment 33 Thorsten Kukuk 2018-04-11 07:14:38 UTC
(In reply to Maximilian Meister from comment #32)

> thanks for the hint, fixed in
> https://github.com/saltstack/salt/pull/46890/commits/
> f4c083dcb8701fc79a0ae5909987758637daed18

This pull request seems to fix the issue. No salt python errors. Still not working due to other bugs in the system I'm looking on now.

Now we only need that fix for Factory and SLE15.
Comment 34 Klaus Kämpf 2018-04-11 07:43:30 UTC
(In reply to Thorsten Kukuk from comment #33)
> (In reply to Maximilian Meister from comment #32)
> 
> > thanks for the hint, fixed in
> > https://github.com/saltstack/salt/pull/46890/commits/
> > f4c083dcb8701fc79a0ae5909987758637daed18
> 
> This pull request seems to fix the issue. No salt python errors. Still not
> working due to other bugs in the system I'm looking on now.
> 
> Now we only need that fix for Factory and SLE15.

We'll add it to 2018.3.0 shortly
Comment 35 Maximilian Meister 2018-04-11 08:02:25 UTC
I wanted to wait until it is merged upstream to backport it right afterwards, just to be sure it won't change anymore
Comment 36 Maximilian Meister 2018-04-11 08:43:59 UTC
(In reply to Thorsten Kukuk from comment #33)
> (In reply to Maximilian Meister from comment #32)
> 
> > thanks for the hint, fixed in
> > https://github.com/saltstack/salt/pull/46890/commits/
> > f4c083dcb8701fc79a0ae5909987758637daed18
> 
> This pull request seems to fix the issue. No salt python errors. Still not
> working due to other bugs in the system I'm looking on now.
> 
> Now we only need that fix for Factory and SLE15.

https://github.com/openSUSE/salt/pull/73
Comment 37 Bo Maryniuk 2018-04-11 09:39:25 UTC
(In reply to Thorsten Kukuk from comment #33)
> (In reply to Maximilian Meister from comment #32)
> 
> > thanks for the hint, fixed in
> > https://github.com/saltstack/salt/pull/46890/commits/
> > f4c083dcb8701fc79a0ae5909987758637daed18
> 
> This pull request seems to fix the issue. No salt python errors. Still not
> working due to other bugs in the system I'm looking on now.
> 
> Now we only need that fix for Factory and SLE15.

Please keep updated here. We've worked together with Maximilian and resolved last problem successfully (good job, Max! :)). I am looking forward to see another pin-pointed error and ready to further help with this.
Comment 38 Thorsten Kukuk 2018-04-11 10:42:17 UTC
(In reply to Bo Maryniuk from comment #37)

> Please keep updated here. We've worked together with Maximilian and resolved
> last problem successfully (good job, Max! :)). I am looking forward to see
> another pin-pointed error and ready to further help with this.

Another problem is bsc#1085632 - salt-api container with python3 backtrace

Not sure if this is "cosmetic" or "critical".
To continue testing with salt I need to fix the admin node setup bugs first, will take some time.
Comment 39 Thorsten Kukuk 2018-04-13 07:19:29 UTC
I fixed the setup scripts, container image names and tags and replaced all python2 with python3 code, so there are no visible errors anymore in the logs.
Except salt-master, where I still get:

07:10:32,109 [INFO    ] Using default for mysql port
07:10:32,109 [INFO    ] Using default for mysql ssl_ca
07:10:32,110 [INFO    ] Using default for mysql ssl_cert
07:10:32,110 [INFO    ] Using default for mysql ssl_key
07:10:32,111 [ERROR   ] Could not store events - returner 'mysql.event_return' raised exception: (2003, "Can't connect to MySQL server on 'localhost' ([Errno 111] Connection refused)")


Don't know who prints the "INFO", the ERROR is comming from PyMySQL. Could it be, that we need to provide the informations for mysql in another way?
Comment 41 Maximilian Meister 2018-04-13 07:55:21 UTC
(In reply to Thorsten Kukuk from comment #39)
> I fixed the setup scripts, container image names and tags and replaced all
> python2 with python3 code, so there are no visible errors anymore in the
> logs.
> Except salt-master, where I still get:
> 
> 07:10:32,109 [INFO    ] Using default for mysql port
> 07:10:32,109 [INFO    ] Using default for mysql ssl_ca
> 07:10:32,110 [INFO    ] Using default for mysql ssl_cert
> 07:10:32,110 [INFO    ] Using default for mysql ssl_key
> 07:10:32,111 [ERROR   ] Could not store events - returner
> 'mysql.event_return' raised exception: (2003, "Can't connect to MySQL server
> on 'localhost' ([Errno 111] Connection refused)")
> 
> 
> Don't know who prints the "INFO", the ERROR is comming from PyMySQL. Could
> it be, that we need to provide the informations for mysql in another way?

the INFO logs most likely come from salt-api -> https://github.com/kubic-project/salt/blob/master/config/master.d/50-api.conf#L1

because we mount the api.conf into the salt-master container for some reason -> https://github.com/kubic-project/caasp-container-manifests/blob/master/manifests/public.yaml#L137

and those configs together lead to this strange behaviour that salt-api logs to salt-master, @kiall did i get that right? i wonder if we can just drop this mount
Comment 42 Jochen Breuer 2018-04-17 17:55:57 UTC
The SR for this patch got accepted this afternoon and you can find the updated RPMs in products:next[0]. It would be nice, if you could you could do a test-drive. Thanks!

[0] https://build.opensuse.org/package/show/systemsmanagement:saltstack:products:next/salt
Comment 43 Jochen Breuer 2018-04-18 11:56:43 UTC
Please let me know if everything works, so that I can also submit this to Factory and SLE15.

I'm not sure who will do this, so I'm setting the need info to you, Maximilian. :p
Comment 46 Thorsten Kukuk 2018-04-27 08:14:37 UTC
Thanks to the fix from Kiall I get now the correct error messages:

pymysql.connect: localhost:3306/salt
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 921, in connect
    **kwargs)
  File "/usr/lib64/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/usr/lib64/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused


So the situation is:
salt/PyMySQL tries to connect to the mysql database via localhost:3306, but this does not work (permission denied). It's not using /run/mysql/mysql.sock

How should this work?
Did we configure salt to use the unix socket, and this information goes somehow lost in salt/PyMySQL (pymysql.connect is called without unix socket name, else
it would not end in this branch), or should mysql listen on localhost:3306?
Comment 47 Thorsten Kukuk 2018-04-27 11:49:18 UTC
Ok, found the problem, comment in our config:

# salt does not support specifying the UNIX socket location here - as a workaround,
 # use the MYSQL_UNIX_PORT environment variable used by libmysqlclient
 # you still need the 'host' value here, or it will use the defaults and try to connect
 # on a host named 'salt'


Problem: PyMySQL does not use libmysqlclient ...
Comment 48 Thorsten Kukuk 2018-04-27 11:56:57 UTC
It's really salt which has no support for a unix socket. the MySQLdb.connect interface supports unix sockets, and according to the MySQLdb documentation, using the socket is even the default and preferred. 

So we need to add support for a unix socket in salt at least for the returner module.
Comment 49 Silvio Moioli 2018-04-30 10:39:33 UTC
(In reply to Thorsten Kukuk from comment #48)
> It's really salt which has no support for a unix socket. the MySQLdb.connect
> interface supports unix sockets, and according to the MySQLdb documentation,
> using the socket is even the default and preferred. 
> 
> So we need to add support for a unix socket in salt at least for the
> returner module.

Thorsten, this sounds like a new bug/feature request to me, are you OK in closing this report?
Comment 50 Thorsten Kukuk 2018-04-30 10:42:34 UTC
(In reply to Silvio Moioli from comment #49)

> Thorsten, this sounds like a new bug/feature request to me, are you OK in
> closing this report?

Yes, this was my plan.
Comment 51 Thorsten Kukuk 2018-04-30 10:49:10 UTC
=> bsc#1091371
Comment 54 Swamp Workflow Management 2018-06-19 19:42:52 UTC
SUSE-SU-2018:1757-1: An update that solves two vulnerabilities and has 20 fixes is now available.

Category: security (moderate)
Bug References: 1059291,1061407,1062464,1064520,1075950,1079048,1081592,1087055,1087278,1087581,1087891,1088888,1089112,1089362,1089526,1090242,1091371,1092161,1092373,1094055,1097174,1097413
CVE References: CVE-2017-14695,CVE-2017-14696
Sources used:
SUSE Manager Tools 12 (src):    salt-2018.3.0-46.28.1, supportutils-plugin-salt-1.1.4-6.9.1
SUSE Manager Server 3.1 (src):    salt-2018.3.0-46.28.1, supportutils-plugin-salt-1.1.4-6.9.1
SUSE Manager Server 3.0 (src):    salt-2018.3.0-46.28.1, supportutils-plugin-salt-1.1.4-6.9.1
SUSE Manager Proxy 3.1 (src):    salt-2018.3.0-46.28.1, supportutils-plugin-salt-1.1.4-6.9.1
SUSE Manager Proxy 3.0 (src):    salt-2018.3.0-46.28.1, supportutils-plugin-salt-1.1.4-6.9.1
SUSE Linux Enterprise Point of Sale 12-SP2 (src):    salt-2018.3.0-46.28.1
SUSE Linux Enterprise Module for Advanced Systems Management 12 (src):    salt-2018.3.0-46.28.1
Comment 55 Swamp Workflow Management 2018-08-20 19:16:52 UTC
SUSE-RU-2018:2462-1: An update that has 23 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1057635,1072599,1087055,1087581,1087891,1089526,1092161,1094055,1095507,1096514,1097174,1097413,1098394,1099323,1099460,1099945,1100142,1100225,1100697,1101812,1101880,1102218,1102265
CVE References: 
Sources used:
SUSE Linux Enterprise Module for Server Applications 15 (src):    salt-2018.3.0-5.9.1
SUSE Linux Enterprise Module for Basesystem 15 (src):    salt-2018.3.0-5.9.1
Comment 56 Swamp Workflow Management 2018-08-24 22:10:18 UTC
openSUSE-RU-2018:2504-1: An update that has 23 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1057635,1072599,1087055,1087581,1087891,1089526,1092161,1094055,1095507,1096514,1097174,1097413,1098394,1099323,1099460,1099945,1100142,1100225,1100697,1101812,1101880,1102218,1102265
CVE References: 
Sources used:
openSUSE Leap 15.0 (src):    salt-2018.3.0-lp150.3.6.1
Comment 58 Swamp Workflow Management 2018-09-25 14:30:38 UTC
This is an autogenerated message for OBS integration:
This bug (1087891) was mentioned in
https://build.opensuse.org/request/show/638205 42.3 / salt
Comment 59 Swamp Workflow Management 2018-10-05 13:24:25 UTC
SUSE-RU-2018:3021-1: An update that solves 5 vulnerabilities and has 125 fixes is now available.

Category: recommended (moderate)
Bug References: 1002529,1004047,1004260,1004723,1008933,1011304,1011800,1012398,1012999,1017078,1019386,1020831,1022562,1022841,1023535,1025896,1027044,1027240,1027722,1030009,1030073,1032213,1032452,1032931,1035914,1036125,1038855,1039370,1040886,1041993,1042749,1043111,1050003,1051948,1052264,1053376,1053955,1057635,1059291,1059758,1060230,1061407,1062462,1062464,1063419,1064520,1065792,1068446,1068566,1071322,1072599,1075950,1079048,1081592,1087055,1087278,1087581,1087891,1088888,1089112,1089362,1089526,1091371,1092161,1092373,1094055,1095507,1095651,1095942,1096514,1097174,1097413,1098394,1099323,1099460,1099945,1100142,1100225,1100697,1101812,1101880,1102013,1102218,1102265,1103530,1103699,1104154,1106164,1108969,849184,849204,849205,955373,958350,959572,963322,965403,967803,969320,970669,971372,972311,972490,975093,975303,975306,975733,975757,976148,978150,978833,979448,979676,980313,983017,983512,985112,985661,986019,988506,989193,989798,990029,990439,990440,991048,993039,993549,996455,999852
CVE References: CVE-2016-1866,CVE-2016-9639,CVE-2017-12791,CVE-2017-14695,CVE-2017-14696
Sources used:
SUSE CaaS Platform 3.0 (src):    salt-2018.3.0-2.3.2, sles12-salt-api-image-3.1.0-3.3.2, sles12-salt-master-image-3.1.0-4.3.2, sles12-salt-minion-image-3.1.0-3.3.2
Comment 60 Swamp Workflow Management 2018-10-17 04:17:35 UTC
openSUSE-RU-2018:3188-1: An update that has 36 recommended fixes can now be installed.

Category: recommended (moderate)
Bug References: 1057635,1072599,1087055,1087278,1087581,1087891,1088888,1089112,1089362,1089526,1091371,1092161,1092373,1094055,1095507,1095651,1095942,1096514,1097174,1097413,1098394,1099323,1099460,1099945,1100142,1100225,1100697,1101812,1101880,1102013,1102218,1102265,1103530,1104154,1106164,1108969
CVE References: 
Sources used:
openSUSE Leap 42.3 (src):    salt-2018.3.0-20.1
Comment 63 Swamp Workflow Management 2019-02-27 17:18:35 UTC
SUSE-OU-2019:13965-1: An update that solves 7 vulnerabilities and has 144 fixes is now available.

Category: optional (low)
Bug References: 1002529,1004047,1004260,1004723,1008933,1011304,1011800,1012398,1012999,1017078,1019386,1020831,1022562,1022841,1023535,1025896,1027044,1027240,1027722,1030009,1030073,1032213,1032452,1032931,1035914,1036125,1038855,1039370,1040886,1041993,1042749,1043111,1050003,1051948,1052264,1053376,1053955,1057635,1059291,1059758,1060230,1061407,1062462,1062464,1063419,1064520,1065792,1068446,1068566,1071322,1072599,1075950,1079048,1081592,1087055,1087278,1087581,1087891,1088888,1089112,1089362,1089526,1091371,1092161,1092373,1094055,1095507,1095651,1095942,1096514,1097174,1097413,1098394,1099323,1099460,1099887,1099945,1100142,1100225,1100697,1101812,1101880,1102013,1102218,1102265,1103530,1104154,1104491,1106164,1107333,1108557,1108834,1108969,1108995,1109893,1110938,1112874,1113698,1113699,1113784,1114029,1114197,1114474,1114824,1116837,1117995,1121091,1123044,1123512,1123865,849184,849204,849205,955373,958350,959572,963322,965403,967803,969320,970669,971372,972311,972490,975093,975303,975306,975733,975757,976148,978150,978833,979448,979676,980313,983017,983512,985112,985661,986019,988506,989193,989798,990029,990439,990440,991048,993039,993549,996455,999852
CVE References: CVE-2016-1866,CVE-2016-9639,CVE-2017-12791,CVE-2017-14695,CVE-2017-14696,CVE-2018-15750,CVE-2018-15751
Sources used:
Comment 64 Swamp Workflow Management 2019-02-27 17:43:05 UTC
SUSE-OU-2019:13964-1: An update that solves 7 vulnerabilities and has 144 fixes is now available.

Category: optional (low)
Bug References: 1002529,1004047,1004260,1004723,1008933,1011304,1011800,1012398,1012999,1017078,1019386,1020831,1022562,1022841,1023535,1025896,1027044,1027240,1027722,1030009,1030073,1032213,1032452,1032931,1035914,1036125,1038855,1039370,1040886,1041993,1042749,1043111,1050003,1051948,1052264,1053376,1053955,1057635,1059291,1059758,1060230,1061407,1062462,1062464,1063419,1064520,1065792,1068446,1068566,1071322,1072599,1075950,1079048,1081592,1087055,1087278,1087581,1087891,1088888,1089112,1089362,1089526,1091371,1092161,1092373,1094055,1095507,1095651,1095942,1096514,1097174,1097413,1098394,1099323,1099460,1099887,1099945,1100142,1100225,1100697,1101812,1101880,1102013,1102218,1102265,1103530,1104154,1104491,1106164,1107333,1108557,1108834,1108969,1108995,1109893,1110938,1112874,1113698,1113699,1113784,1114029,1114197,1114474,1114824,1116837,1117995,1121091,1123044,1123512,1123865,849184,849204,849205,955373,958350,959572,963322,965403,967803,969320,970669,971372,972311,972490,975093,975303,975306,975733,975757,976148,978150,978833,979448,979676,980313,983017,983512,985112,985661,986019,988506,989193,989798,990029,990439,990440,991048,993039,993549,996455,999852
CVE References: CVE-2016-1866,CVE-2016-9639,CVE-2017-12791,CVE-2017-14695,CVE-2017-14696,CVE-2018-15750,CVE-2018-15751
Sources used:
Comment 69 Swamp Workflow Management 2020-06-23 16:27:10 UTC
SUSE-SU-2020:14402-1: An update that solves 11 vulnerabilities and has 245 fixes is now available.

Category: security (moderate)
Bug References: 1002529,1003449,1004047,1004260,1004723,1008933,1011304,1011800,1012398,1012999,1013876,1013938,1015882,1017078,1019386,1020831,1022562,1022841,1023535,1024406,1025896,1027044,1027240,1027426,1027722,1030009,1030073,1032213,1032452,1032931,1035914,1036125,1038855,1039370,1040886,1041993,1042749,1043111,1044719,1050003,1051948,1052264,1053376,1053955,1057635,1059291,1059758,1060230,1061407,1062462,1062464,1063419,1064520,1065792,1068446,1068566,1070372,1071322,1072599,1075950,1076578,1079048,1080290,1081151,1081592,1083294,1085667,1087055,1087278,1087581,1087891,1088070,1088888,1089112,1089362,1089526,1091371,1092161,1092373,1094055,1094190,1095507,1095651,1095942,1096514,1097174,1097413,1098394,1099323,1099460,1099887,1099945,1100142,1100225,1100697,1101780,1101812,1101880,1102013,1102218,1102265,1102819,1103090,1103530,1103696,1104034,1104154,1104491,1106164,1107333,1108557,1108834,1108969,1108995,1109023,1109893,1110938,1111542,1112874,1113698,1113699,1113784,1114029,1114197,1114474,1114824,1116343,1116837,1117995,1121091,1121439,1122663,1122680,1123044,1123512,1123865,1124277,1125015,1125610,1125744,1127389,1128061,1128554,1129079,1129243,1130077,1130588,1130784,1131114,1132076,1133523,1133647,1134860,1135360,1135507,1135567,1135732,1135881,1137642,1138454,1139761,1140193,1140912,1143301,1146192,1146382,1148311,1148714,1150447,1151650,1151947,1152366,1153090,1153277,1153611,1154620,1154940,1155372,1157465,1157479,1158441,1159284,1162327,1162504,1163871,1163981,1165425,1165572,1167437,1167556,1168340,1169604,1169800,1170104,1170288,1170595,1171687,1171906,1172075,1173072,769106,769108,776615,849184,849204,849205,879904,887879,889605,892707,902494,908849,926318,932288,945380,948245,955373,958350,959572,963322,965403,967803,969320,970669,971372,972311,972490,975093,975303,975306,975733,975757,976148,977264,978150,978833,979448,979676,980313,983017,983512,985112,985661,986019,987798,988506,989193,989798,990029,990439,990440,991048,993039,993549,996455,999852
CVE References: CVE-2016-1866,CVE-2016-9639,CVE-2017-12791,CVE-2017-14695,CVE-2017-14696,CVE-2018-15750,CVE-2018-15751,CVE-2019-17361,CVE-2019-18897,CVE-2020-11651,CVE-2020-11652
Sources used:

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 71 Swamp Workflow Management 2020-07-21 04:24:50 UTC
SUSE-SU-2020:14431-1: An update that solves 11 vulnerabilities and has 251 fixes is now available.

Category: security (moderate)
Bug References: 1002529,1003449,1004047,1004260,1004723,1008933,1011304,1011800,1012398,1012999,1013876,1013938,1015882,1017078,1019386,1020831,1022562,1022841,1023535,1024406,1025896,1027044,1027240,1027426,1027722,1030009,1030073,1032213,1032452,1032931,1035914,1036125,1038855,1039370,1040886,1041993,1042749,1043111,1044719,1050003,1051948,1052264,1053376,1053955,1057635,1059291,1059758,1060230,1061407,1062462,1062464,1063419,1064520,1065792,1068446,1068566,1070372,1071322,1072599,1075950,1076578,1079048,1080290,1081151,1081592,1083294,1085667,1087055,1087278,1087581,1087891,1088070,1088888,1089112,1089362,1089526,1091371,1092161,1092373,1094055,1094190,1095507,1095651,1095942,1096514,1097174,1097413,1098394,1099323,1099460,1099887,1099945,1100142,1100225,1100697,1101780,1101812,1101880,1102013,1102218,1102265,1102819,1103090,1103530,1103696,1104034,1104154,1104491,1106164,1107333,1108557,1108834,1108969,1108995,1109023,1109893,1110938,1111542,1112874,1113698,1113699,1113784,1114029,1114197,1114474,1114824,1116343,1116837,1117995,1121091,1121439,1122663,1122680,1123044,1123512,1123865,1124277,1125015,1125610,1125744,1127389,1128061,1128554,1129079,1129243,1130077,1130588,1130784,1131114,1132076,1133523,1133647,1134860,1135360,1135507,1135567,1135656,1135732,1135881,1137642,1138454,1138952,1139761,1140193,1140912,1143301,1146192,1146382,1148311,1148714,1150447,1151650,1151947,1152366,1153090,1153277,1153611,1154620,1154940,1155372,1157465,1157479,1158441,1158940,1159118,1159284,1160931,1162327,1162504,1163871,1165425,1165572,1167437,1167556,1168340,1169604,1169800,1170042,1170104,1170288,1170595,1171687,1171906,1172075,1173072,1174165,769106,769108,776615,849184,849204,849205,879904,887879,889605,892707,902494,908849,926318,932288,945380,948245,955373,958350,959572,963322,965403,967803,969320,970669,971372,972311,972490,975093,975303,975306,975733,975757,976148,977264,978150,978833,979448,979676,980313,983017,983512,985112,985661,986019,987798,988506,989193,989798,990029,990439,990440,991048,993039,993549,996455,999852
CVE References: CVE-2016-1866,CVE-2016-9639,CVE-2017-12791,CVE-2017-14695,CVE-2017-14696,CVE-2018-15750,CVE-2018-15751,CVE-2019-17361,CVE-2019-18897,CVE-2020-11651,CVE-2020-11652
Sources used:

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 77 Swamp Workflow Management 2021-02-08 14:45:14 UTC
SUSE-SU-2021:0315-1: An update that solves 14 vulnerabilities and has 218 fixes is now available.

Category: security (moderate)
Bug References: 1002529,1004047,1004260,1004723,1008933,1011304,1011800,1012398,1012999,1017078,1019386,1020831,1022562,1022841,1023535,1025896,1027044,1027240,1027722,1030009,1030073,1032213,1032452,1032931,1035914,1036125,1038855,1039370,1040886,1041993,1042749,1043111,1050003,1051948,1052264,1053376,1053955,1057635,1059291,1059758,1060230,1061407,1062462,1062464,1063419,1064520,1065792,1068446,1068566,1071322,1072599,1075950,1079048,1081592,1083110,1087055,1087278,1087581,1087891,1088888,1089112,1089362,1089526,1091371,1092161,1092373,1094055,1095507,1095651,1095942,1096514,1097174,1097413,1098394,1099323,1099460,1099887,1099945,1100142,1100225,1100697,1101780,1101812,1101880,1102013,1102218,1102248,1102265,1102819,1103530,1104154,1104491,1106164,1107333,1108557,1108834,1108969,1108995,1109893,1110938,1112874,1113698,1113699,1113784,1114029,1114197,1114474,1114824,1116343,1116837,1117995,1121091,1121439,1122663,1122680,1123044,1123512,1123865,1124277,1125015,1128061,1128554,1129079,1130588,1130784,1131114,1132076,1133523,1133647,1134860,1135360,1135507,1135567,1135656,1135732,1137642,1138952,1139761,1140193,1140912,1143301,1146192,1146382,1148714,1150447,1151650,1151947,1152366,1153611,1154620,1157465,1157479,1158441,1158940,1159118,1159284,1159670,1160931,1162327,1162504,1165425,1165572,1167437,1167556,1168340,1169604,1169800,1170042,1170104,1170288,1170595,1171461,1171906,1172075,1172211,1173072,1173909,1173911,1173936,1174165,1175549,1175987,1176024,1176294,1176397,1176480,1177867,1178319,1178361,1178362,1178485,849184,849204,849205,955373,958350,959572,963322,965403,967803,969320,970669,971372,972311,972490,975093,975303,975306,975733,975757,976148,978150,978833,979448,979676,980313,983017,983512,985112,985661,986019,988506,989193,989798,990029,990439,990440,991048,993039,993549,996455,999852
CVE References: CVE-2016-1866,CVE-2016-9639,CVE-2017-12791,CVE-2017-14695,CVE-2017-14696,CVE-2018-15750,CVE-2018-15751,CVE-2019-17361,CVE-2019-18897,CVE-2020-11651,CVE-2020-11652,CVE-2020-16846,CVE-2020-17490,CVE-2020-25592
JIRA References: 
Sources used:

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 78 Swamp Workflow Management 2021-02-08 15:20:27 UTC
SUSE-SU-2021:0316-1: An update that solves 14 vulnerabilities and has 218 fixes is now available.

Category: security (moderate)
Bug References: 1002529,1004047,1004260,1004723,1008933,1011304,1011800,1012398,1012999,1017078,1019386,1020831,1022562,1022841,1023535,1025896,1027044,1027240,1027722,1030009,1030073,1032213,1032452,1032931,1035914,1036125,1038855,1039370,1040886,1041993,1042749,1043111,1050003,1051948,1052264,1053376,1053955,1057635,1059291,1059758,1060230,1061407,1062462,1062464,1063419,1064520,1065792,1068446,1068566,1071322,1072599,1075950,1079048,1081592,1083110,1087055,1087278,1087581,1087891,1088888,1089112,1089362,1089526,1091371,1092161,1092373,1094055,1095507,1095651,1095942,1096514,1097174,1097413,1098394,1099323,1099460,1099887,1099945,1100142,1100225,1100697,1101780,1101812,1101880,1102013,1102218,1102248,1102265,1102819,1103530,1104154,1104491,1106164,1107333,1108557,1108834,1108969,1108995,1109893,1110938,1112874,1113698,1113699,1113784,1114029,1114197,1114474,1114824,1116343,1116837,1117995,1121091,1121439,1122663,1122680,1123044,1123512,1123865,1124277,1125015,1128061,1128554,1129079,1130588,1130784,1131114,1132076,1133523,1133647,1134860,1135360,1135507,1135567,1135656,1135732,1137642,1138952,1139761,1140193,1140912,1143301,1146192,1146382,1148714,1150447,1151650,1151947,1152366,1153611,1154620,1157465,1157479,1158441,1158940,1159118,1159284,1159670,1160931,1162327,1162504,1165425,1165572,1167437,1167556,1168340,1169604,1169800,1170042,1170104,1170288,1170595,1171461,1171906,1172075,1172211,1173072,1173909,1173911,1173936,1174165,1175549,1175987,1176024,1176294,1176397,1176480,1177867,1178319,1178361,1178362,1178485,849184,849204,849205,955373,958350,959572,963322,965403,967803,969320,970669,971372,972311,972490,975093,975303,975306,975733,975757,976148,978150,978833,979448,979676,980313,983017,983512,985112,985661,986019,988506,989193,989798,990029,990439,990440,991048,993039,993549,996455,999852
CVE References: CVE-2016-1866,CVE-2016-9639,CVE-2017-12791,CVE-2017-14695,CVE-2017-14696,CVE-2018-15750,CVE-2018-15751,CVE-2019-17361,CVE-2019-18897,CVE-2020-11651,CVE-2020-11652,CVE-2020-16846,CVE-2020-17490,CVE-2020-25592
JIRA References: 
Sources used:

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.