Bug 1083830 - ffado fails to build with Python3 scons
ffado fails to build with Python3 scons
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Development
Current
Other openSUSE Factory
: P5 - None : Normal (vote)
: ---
Assigned To: Dave Plater
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-03-03 09:58 UTC by Dave Plater
Modified: 2020-04-09 20:05 UTC (History)
3 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dave Plater 2018-03-03 09:58:28 UTC
I've been working with the ffado developer to try to fix the scons with python3 issues and already in ffado-2.4.0 there are three scons functions that fail with python3. With the upcoming 2.4.1 release ffado no longer builds due to more failed python3 scons functions.
Fedora has already reverted to python2.
I've attempted to build scons git which contains many python 3 fixes but failed.
Comment 1 Andreas Stieger 2018-03-04 12:42:28 UTC
We fixed up scons 3.0.1 for most of the Python 3 issues, e.g. in the libserf consumers. Note that scons itself may have issues, but also the SConstruct as used by ffado.

I do not think that python 2 will remain available in Tumbleweed, so this is not really a practical avenue. Any new scons releases will of course be bumped, not sure what else we can do?
Comment 2 Dave Plater 2018-03-05 06:14:33 UTC
If somebody with more knowledge of building scons could build a git snapshot, I think that this might solve the issue. openSUSE is now the only distribution building scons 3.0.1 with python3.
See:
https://build.opensuse.org/package/live_build_log/home:plater/ffado/openSUSE_Tumbleweed/x86_64

and:
https://build.opensuse.org/package/show/home:plater/scons
For my attempts at building git snapshot.
Comment 3 Andreas Stieger 2018-03-05 14:36:35 UTC
So what exactly is the ffado problems, and why do they not fix it in ffado?
Comment 4 Andreas Stieger 2018-03-05 14:44:00 UTC
Are you reporting this against these failures?
https://build.opensuse.org/package/live_build_log/home:plater/ffado/openSUSE_Tumbleweed/x86_64

[  194s] building 'libffado/ffado.h' from 'libffado/ffado.h.in'
[  194s] scons: *** [version.h] TypeError : sequence item 1: expected str instance, bytes found
[  194s] Traceback (most recent call last):
[  194s]   File "/usr/lib/scons-3.0.1/SCons/Action.py", line 689, in __call__
[  194s]     cmd = self.strfunction(target, source, env, executor)
[  194s] TypeError: CacheRetrieveString() takes 3 positional arguments but 4 were given
[  194s] 
[  194s] During handling of the above exception, another exception occurred:
[  194s] 
[  194s] Traceback (most recent call last):
[  194s]   File "/usr/lib/scons-3.0.1/SCons/Taskmaster.py", line 241, in execute
[  194s]     if not t.retrieve_from_cache():
[  194s]   File "/usr/lib/scons-3.0.1/SCons/Node/FS.py", line 2925, in retrieve_from_cache
[  194s]     return self.get_build_env().get_CacheDir().retrieve(self)
[  194s]   File "/usr/lib/scons-3.0.1/SCons/CacheDir.py", line 263, in retrieve
[  194s]     if CacheRetrieve(node, [], env, execute=1) == 0:
[  194s]   File "/usr/lib/scons-3.0.1/SCons/Action.py", line 691, in __call__
[  194s]     cmd = self.strfunction(target, source, env)
[  194s]   File "/usr/lib/scons-3.0.1/SCons/CacheDir.py", line 66, in CacheRetrieveString
[  194s]     cachedir, cachefile = cd.cachepath(t)
[  194s]   File "/usr/lib/scons-3.0.1/SCons/CacheDir.py", line 224, in cachepath
[  194s]     sig = node.get_cachedir_bsig()
[  194s]   File "/usr/lib/scons-3.0.1/SCons/Node/FS.py", line 3391, in get_cachedir_bsig
[  194s]     result = self.cachesig = SCons.Util.MD5collect(sigs)
[  194s]   File "/usr/lib/scons-3.0.1/SCons/Util.py", line 1550, in MD5collect
[  194s]     return MD5signature(', '.join(signatures))
[  194s] TypeError: sequence item 1: expected str instance, bytes found
[  194s] scons: building terminated because of errors.
[  194s] error: Bad exit status from /var/tmp/rpm-tmp.KmlRYb (%build)
Comment 5 Andreas Stieger 2018-03-05 22:01:56 UTC
Also you omitted the fact that you had libffado-SConstruct-py3.patch in your package.

Can you please avoid making the bug about the proposed solution: Let's talk about the underlying problems: scons provides the full python power in build scripts, so not every conceivable build script will work on Python 3. In this case this affects ffado.

We seem to not have Python 2 available in Tumbleweed anymore, so this is not a matter of switching back.

Feedback from scons developers is that this is a problem with the build scripts consuming cli output, which is python 3 is no longer a string but bytes / utf-8, needing a conversion.
Comment 6 Dave Plater 2018-03-06 05:43:13 UTC
(In reply to Andreas Stieger from comment #5)
> Also you omitted the fact that you had libffado-SConstruct-py3.patch in your
> package.
> 
> Can you please avoid making the bug about the proposed solution: Let's talk
> about the underlying problems: scons provides the full python power in build
> scripts, so not every conceivable build script will work on Python 3. In
> this case this affects ffado.
> 
> We seem to not have Python 2 available in Tumbleweed anymore, so this is not
> a matter of switching back.
> 
> Feedback from scons developers is that this is a problem with the build
> scripts consuming cli output, which is python 3 is no longer a string but
> bytes / utf-8, needing a conversion.

This is the only remaining part of the original Fedora patch:

"return True # FIXME: The following TryRun fails on Python3"

TryRun() is an integral part of scons, it uses TryBuild() which fails on python3 along with CheckLibWithHeader(), this is an scons bug, if you look at the patch libffado-SConstruct-py3.patch you will see that it works around these problems. These functions are used as described in the scons documentation therefore IMHO it is a bug.
If there is a link that I can pass on to the ffado developer on how to overcome these problems, that will help. He is aware of decode() functions.
Comment 7 Andreas Stieger 2018-03-06 07:31:05 UTC
(In reply to Dave Plater from comment #6)
> TryRun() is an integral part of scons, it uses TryBuild() which fails on
> python3 along with CheckLibWithHeader(), this is an scons bug, if you look
> at the patch libffado-SConstruct-py3.patch you will see that it works around
> these problems. These functions are used as described in the scons
> documentation therefore IMHO it is a bug.

Yes but why is this not in your original bug? I encountered similar problems and fixed them both with scons 3.0.1 and in the SConstruct of serf which used python 2 specific code.

So the with ffado+scons+python 3 is confirmable. Happy to patch in anything in scons git that you think helps here.
Comment 8 Dave Plater 2018-03-06 10:38:35 UTC
Suffixing ".decode('utf-8')" doesn't work with either:
def CompilerCheck( context ):
    context.Message( "Checking for a working C-compiler " )
    ret = context.TryRun( """
#include <stdio.h>

int main() {
    printf( "Hello World!" );
    return 0;
}""", '.c' ).decode('utf-8')[0]
    context.Result( ret )

or

    if 'CFLAGS' in env:
        oldcf = env['CFLAGS']
    else:
        oldcf = ""
    env.Append(CFLAGS = '-std=c99')
    if conf.CheckLibWithHeader( "m", "math.h", "c", "lrint(3.2);" ).decode('utf-8'):
        HAVE_LRINT = 1
    else:
        HAVE_LRINT = 0
    if conf.CheckLibWithHeader( "m", "math.h", "c", "lrintf(3.2);" ).decode('utf-8'):
        HAVE_LRINTF = 1
    else:
        HAVE_LRINTF = 0
    env['HAVE_LRINT'] = HAVE_LRINT;
    env['HAVE_LRINTF'] = HAVE_LRINTF;
    env.Replace(CFLAGS=oldcf)

What am I doing wrong?
The ffado developer doesn't have the facility to run python 3 scons and relies on me to supply the build failures.
Comment 9 Dave Plater 2018-03-12 08:23:06 UTC
I've joined the scons user email list and so far we have a small patch that passes TryRun and CheckLibWithHeader.
Comment 10 Dave Plater 2018-03-15 09:57:07 UTC
I've so far got ffado-2.4.1 to build by limiting the jobs to 1.
I've submitted patched scons via sr#587455 I need to wait for this to be in factory before submitting ffado as I have no wish to patch out CheckLibWithHeader() which is fixed with this patch.
Comment 11 Andreas Stieger 2018-03-15 09:58:28 UTC
forwarded sr#587455
Comment 12 Swamp Workflow Management 2018-03-15 10:30:20 UTC
This is an autogenerated message for OBS integration:
This bug (1083830) was mentioned in
https://build.opensuse.org/request/show/587460 Factory / scons
Comment 13 Dave Plater 2018-03-15 14:30:48 UTC
This is still work in progress. It's assigned to me anyway.
Comment 14 Andreas Stieger 2018-03-16 07:46:46 UTC
From https://build.opensuse.org/request/show/587460
"Seems to break gpsd"
"gpsd build fail... does not even detect socket.h"
Comment 15 Dave Plater 2019-01-02 14:24:20 UTC
I've received a christmas present called scons-3.0.2 which resolves this bug.
Comment 16 Swamp Workflow Management 2020-03-27 06:23:14 UTC
This is an autogenerated message for OBS integration:
This bug (1083830) was mentioned in
https://build.opensuse.org/request/show/788740 15.1+Backports:SLE-15-SP1 / scons