Bug 1199388 - release-notes-openSUSE fail to build
release-notes-openSUSE fail to build
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Distribution
Classification: openSUSE
Component: Documentation
Leap 15.4
Other Other
: P1 - Urgent : Critical (vote)
: ---
Assigned To: Frank Sundermeyer
Frank Sundermeyer
:
Depends on: 1199249
Blocks:
  Show dependency treegraph
 
Reported: 2022-05-10 11:44 UTC by Lubos Kocman
Modified: 2022-05-13 11:05 UTC (History)
3 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---
lubos.kocman: SHIP_STOPPER+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Lubos Kocman 2022-05-10 11:44:53 UTC
openSUSE Leap 15.4 release notes are failing to build starting by this week
https://build.opensuse.org/package/live_build_log/openSUSE:Leap:15.4/release-notes-openSUSE/standard/x86_64

After initial lookup Max thinks it could be related to the week old update of daps in SUSE:SLE-15-SP3:Update
Comment 1 Lubos Kocman 2022-05-10 11:46:02 UTC
[  488s]             MAKE_CMD: /usr/bin/make -f /usr/share/daps/make/selector.mk -j8 -s text PROFCONDITION=general;beta
[  488s]        XSLTPROCESSOR: /usr/bin/xsltproc
[  488s]    XML_CATALOG_FILES: /etc/xml/catalog
[  488s] 
[  488s] ---------------
[  488s] 
[  489s]    Creating temporary single HTML page
[  489s] No "sv" localization of "version" exists; using "en".
[  489s] No "sv" localization of "version" exists; using "en".
[  489s]    Creating ASCII file
[  489s] Find the TEXT book at:
[  489s] /home/abuild/rpmbuild/BUILD/release-notes-openSUSE-15.4.20220228/build/release-notes.sv/release-notes.sv.txt
[  489s] lang=`echo build/release-notes.zh_CN/release-notes.zh_CN.txt | awk -F '.' '{gsub("/.*","",$2); print($2)}'` ; \
[  489s] LANG=${lang} daps -vv --styleroot /usr/share/xml/docbook/stylesheet/opensuse2013-ns -m xml/release-notes.${lang}.xml text \
[  489s] PROFCONDITION="general\;beta"
[  490s] [1;31mFatal error:[0m
[  490s] [1;31m[0m
[  490s] make: *** [Makefile:160: build/release-notes.zh_CN/release-notes.zh_CN.txt] Error 1
[  490s] rm po/fi.mo po/cs.mo po/id.mo po/ja.mo po/zh_CN.mo po/es.mo po/it.mo po/fr.mo po/nl.mo po/sk.mo po/pt_BR.mo po/zh_TW.mo po/sv.mo po/ca.mo po/ru.mo po/de.mo
[  490s] error: Bad exit status from /var/tmp/rpm-tmp.tyroaX (%build)
[  490s] 
[  490s] 
[  490s] RPM build errors:
[  490s]     Bad exit status from /var/tmp/rpm-tmp.tyroaX (%build)
[  490s] ### VM INTERACTION START ###
Comment 2 Lubos Kocman 2022-05-10 11:58:32 UTC
Seems to be related to Bug 1199249
Comment 3 Lubos Kocman 2022-05-10 13:00:29 UTC
From Frank:

the SLE RT issue was a different one
previous versions of DAPS work, because we are doing the validation differently than with newer DAPS versions
the error appears when running a python script that is part of the DAPS package (/usr/share/daps/libexec/daps-xmlwellformed)
the error only occurs when setting the LANG environment variable to a two byte language (tested with zh_CN, zh_TW, ja_JP, ko_KR)
the error does not occur when running the script with the very same parameters and data, but LANG set to a non two byte language (tested with de_DE. en_UK, pt_BR)
the error is independent from the XML data fed into the script, it solely depends on the LANG variable
the python script does not exit with an error, but with a segmentation fault, This would not be the case if it was a simple error in the script
> LANG=ja_jp /usr/share/daps/libexec/daps-xmlwellformed --xinclude /tmp/release-notes-openSUSE-15.4.20220228/xml/release-notes.en.xml
[1]    1906 segmentation fault (core dumped)  LANG=ja_jp /usr/share/daps/libexec/daps-xmlwellformed --xinclude 
The very same command works without problems when you run it with LANG=pt_BR  (tested on Tumbleweed, the same happens when building on Leap 15.4)
Could someone please test whether the command above works on Leap 15.3 or Leap 15.2?
If so, that would be proof that this error is caused by a regression in one of the python packages
Comment 4 Tim Serong 2022-05-11 04:21:14 UTC
(In reply to Lubos Kocman from comment #3)
> Could someone please test whether the command above works on Leap 15.3 or
> Leap 15.2?

On Leap 15.2:

 > rpm -q daps
 daps-3.0.0-lp152.4.2.noarch
 > LANG=ja_jp /usr/share/daps/libexec/daps-xmlwellformed \
     --xinclude release-notes-openSUSE-15.4.20220228/xml/release-notes.xml
 > echo $?
 0

On Leap 15.4 (in an osc build chroot, but, close enough):

 > rpm -q daps
 daps-3.3.1-150300.14.3.2.noarch
 > LANG=ja_jp /usr/share/daps/libexec/daps-xmlwellformed \
     --xinclude release-notes-openSUSE-15.4.20220228/xml/release-notes.xml
 Segmentation fault (core dumped)

On Tumbleweed:

 > rpm -q daps
 daps-3.3.1-173.2.noarch
 > LANG=ja_jp /usr/share/daps/libexec/daps-xmlwellformed \
     --xinclude release-notes-openSUSE-15.4.20220228/xml/release-notes.xml
 > echo $?
 0
Comment 5 Tim Serong 2022-05-11 04:33:27 UTC
Isn't it odd though, that it builds Japanese, Chinese, etc. PDFs just fine, but only seems to die when building Chinese plain text?
Comment 6 Tim Serong 2022-05-11 06:19:28 UTC
Ooooh, look at this in the Makefile:

> pdf: $(PDF_FILES)
> $(PDF_FILES): po/LINGUAS translatedxml
>         lang=$(LANG_COMMAND) ; \
>         $(DAPS_COMMAND) pdf PROFCONDITION="general\;$(LIFECYCLE)" \
>         --param "generate.consistent.ids=1"
> 
> single-html: $(SINGLE_HTML_FILES)
> $(SINGLE_HTML_FILES): po/LINGUAS translatedxml
>         lang=$(LANG_COMMAND) ; \
>         $(DAPS_COMMAND) html --single \
>         --stringparam "homepage='https://www.opensuse.org'" \
>         --param "generate.consistent.ids=1" \
>         PROFCONDITION="general\;$(LIFECYCLE)"
> 
> yast-html: | $(DIRS) $(YAST_HTML_FILES)
> $(YAST_HTML_FILES): po/LINGUAS $(YAST_PROFILED_FILES)
>         lang=$(LANG_COMMAND) ; \
>           $(XSLTPROC_COMMAND) /usr/share/daps/daps-xslt/relnotes/yast.xsl build/.profiled/general_$(LIFECYCLE)/release-notes.$${lang}.xml > $@
> 
> text: $(TXT_FILES)
> $(TXT_FILES): po/LINGUAS translatedxml
>         lang=$(LANG_COMMAND) ; \
>         LANG=$${lang} $(DAPS_COMMAND) text \    <<<<<<<<<< LANG IS SET HERE
>         PROFCONDITION="general\;$(LIFECYCLE)"
> 

The LANG environment variable isn't set at all for PDF or HTML generation.  It's only set when generating plain text.  So that's why the other formats succeed and text fails - they don't hit the segfault because no LANG was specified.  Is that intentional?  Can we "fix" this by simply getting rid of LANG? i.e.:

> --- release-notes-openSUSE-15.4.20220228.orig/Makefile
> +++ release-notes-openSUSE-15.4.20220228/Makefile
> @@ -158,7 +158,7 @@ $(YAST_PROFILED_FILES): po/LINGUAS trans
>  text: $(TXT_FILES)
>  $(TXT_FILES): po/LINGUAS translatedxml
>  	lang=$(LANG_COMMAND) ; \
> -	LANG=$${lang} $(DAPS_COMMAND) text \
> +	$(DAPS_COMMAND) text \
>  	PROFCONDITION="general\;$(LIFECYCLE)"
>  
>  $(DIRS):

It *does* build with that patch applied.  I just don't know if it's correct to make that change.

As for the segfault, I've got some more information about that.  I discovered it *didn't* happen when running as the root user in my build chroot, but it *did* happen when running as the abuild user (which is what the build actually runs as).  Turns out root's environment has LC_CTYPE=en_US.utf8, whereas for abuild, that's not set.  If I make the following change, the build works, with LANG still set:

> --- release-notes-openSUSE-15.4.20220228.orig/Makefile
> +++ release-notes-openSUSE-15.4.20220228/Makefile
> @@ -158,7 +158,7 @@ $(YAST_PROFILED_FILES): po/LINGUAS trans
>  text: $(TXT_FILES)
>  $(TXT_FILES): po/LINGUAS translatedxml
>  	lang=$(LANG_COMMAND) ; \
> -	LANG=$${lang} $(DAPS_COMMAND) text \
> +	LC_CTYPE=$${lang}.utf8 LANG=$${lang} $(DAPS_COMMAND) text \
>  	PROFCONDITION="general\;$(LIFECYCLE)"

So maybe that's a better workaround. I _assume_ "LC_CTYPE=$${lang}.utf8" is the way to go here, but I'm not 100% certain, as I'm not an expert in how these environment variables interact.

Also, it turns out this *is* broken on Tumbleweed.  Try this:

 > unset LC_CTYPE
 > LANG=ja_jp /usr/share/daps/libexec/daps-xmlwellformed \
     --xinclude release-notes-openSUSE-15.4.20220228/xml/release-notes.xml
 Segmentation fault (core dumped)

To debug the segfault further, I did this:

> --- /usr/share/daps/libexec/daps-xmlwellformed.orig	2022-05-11 05:49:01.556537094 +0000
> +++ /usr/share/daps/libexec/daps-xmlwellformed	2022-05-11 05:49:11.068460845 +0000
> @@ -22,6 +22,8 @@
>  
>  from lxml import etree
>  
> +import faulthandler
> +faulthandler.enable()
>  
>  ROOTLOGGER = "lxmlerrors"
>  # Hint: file keyword needs to be filled with extra keyword on the

Then got this:

 > LANG=ja_jp /usr/share/daps/libexec/daps-xmlwellformed      --xinclude xml/release-notes.xml
 Fatal Python error: Segmentation fault
 
 Current thread 0x00007f0655034740 (most recent call first):
   File "/usr/share/daps/libexec/daps-xmlwellformed", line 131 in check_wellformedness
   File "/usr/share/daps/libexec/daps-xmlwellformed", line 215 in main
   File "/usr/share/daps/libexec/daps-xmlwellformed", line 235 in <module>
 Segmentation fault (core dumped)

Which is a call to python-lxml's etree.  Whether this is a bug in the python wrapper or the underlying libxml2, I don't know.
Comment 7 Lukas Kucharczyk 2022-05-11 08:46:42 UTC
I've tried unsetting the LANG variable but the resulting file contains garbled text (checked for Chinese). However, adding LC_CTYPE in front of LANG as you did results in a valid text and no error!
Comment 8 Tim Serong 2022-05-11 09:07:37 UTC
(In reply to Lukas Kucharczyk from comment #7)
> I've tried unsetting the LANG variable but the resulting file contains
> garbled text (checked for Chinese). However, adding LC_CTYPE in front of
> LANG as you did results in a valid text and no error!

Excellent :-D
Comment 9 Lukas Kucharczyk 2022-05-11 12:28:35 UTC
Fixed in https://github.com/openSUSE/release-notes-openSUSE/commit/17f1214ee5123aeb90687e77e1f56cdd1beb3361, I will wait for Frank to let us know whether this is an actual fix or just a temporary workaround
Comment 10 OBSbugzilla Bot 2022-05-11 14:40:47 UTC
This is an autogenerated message for OBS integration:
This bug (1199388) was mentioned in
https://build.opensuse.org/request/show/976384 15.4 / release-notes-openSUSE
Comment 11 Lubos Kocman 2022-05-13 11:05:18 UTC
Marking as resolved