Bug 1119396 - Enable php7 testsuite
Enable php7 testsuite
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Distribution
Classification: openSUSE
Component: Other
Leap 15.0
Other Other
: P5 - None : Normal (vote)
: ---
Assigned To: Martin Pluskal
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-12-13 13:49 UTC by Martin Pluskal
Modified: 2021-09-14 12:47 UTC (History)
1 user (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Pluskal 2018-12-13 13:49:16 UTC
Lets enable php7 testsuite to be executed during build time:
* lets check build times first though
* compress and bundle test log
Comment 1 Martin Pluskal 2018-12-13 13:49:51 UTC
Local build time with testsuite enabled (8 core cpu) is ~40 minutes
Comment 2 Martin Pluskal 2018-12-14 08:58:44 UTC
And in OBS, build times are ~ 4 hours
Comment 3 Petr Gajdos 2018-12-18 07:41:52 UTC
I think this is not a good idea, at least without a bit more invention.

(1) build times as you already stated; in your project it is ~ 1.4 h:
[ 1419s] + make test
[..]
[ 4819s] WARNED TEST SUMMARY
So the build is three times
That can be overcome by osc build switch while making testing 

(2) 
$ osc rbl home:pluskalm:branches:devel:languages:php php7 openSUSE_Factory x86_64 | grep '[^X]FAIL ' | wc -l
120
$

(3) most important: results of certain amount of tests is not reliable, ask QAM

(4) as (2) suggests, with the patch you are sending me you are actually not evaluate the test suite results, so running it so is pointlessly waste of build time:
[..]
make test || true
[..]
To be complete, you can find the reason why this code is in %check in build-test.sh along php7 sources.
Comment 4 Petr Gajdos 2018-12-18 07:48:32 UTC
Obviously I forgot to finish even two sentences:

(In reply to Petr Gajdos from comment #3)
> So the build is three times
  longer than without the test suite.
> That can be overcome by osc build switch while making testing 
  build, yes. Not sure there are another consequences in IBS/OBS.
Comment 5 Martin Pluskal 2018-12-18 07:57:35 UTC
(In reply to Petr Gajdos from comment #3)
> I think this is not a good idea, at least without a bit more invention.
> 
> (1) build times as you already stated; in your project it is ~ 1.4 h:
> [ 1419s] + make test
> [..]
> [ 4819s] WARNED TEST SUMMARY
> So the build is three times
> That can be overcome by osc build switch while making testing 
> 
> (2) 
> $ osc rbl home:pluskalm:branches:devel:languages:php php7 openSUSE_Factory
> x86_64 | grep '[^X]FAIL ' | wc -l
> 120
> $
> 
> (3) most important: results of certain amount of tests is not reliable, ask
> QAM
I am in QAM :)
> 
> (4) as (2) suggests, with the patch you are sending me you are actually not
> evaluate the test suite results, so running it so is pointlessly waste of
> build time:
True, but results can be evaluated later - by QAM
> [..]
> make test || true
> [..]
> To be complete, you can find the reason why this code is in %check in
> build-test.sh along php7 sources.

Alternative to doing this would be to carve out tests as is done in qa_test_php and have some package like i.e php7-test, that could be then run by QA* when needed - would that make more sense to you?
Comment 6 Petr Gajdos 2018-12-18 08:35:53 UTC
(In reply to Martin Pluskal from comment #5)
> I am in QAM :)

You indeed still are? I really have to go trough our org table in the kitchen again.
 
> > (4) as (2) suggests, with the patch you are sending me you are actually not
> > evaluate the test suite results, so running it so is pointlessly waste of
> > build time:
> True, but results can be evaluated later - by QAM

Then I think it should be noted as a bold comment, preferably exactly above make test || true to not waste time of people in the future.

> Alternative to doing this would be to carve out tests as is done in
> qa_test_php and have some package like i.e php7-test, that could be then run
> by QA* when needed - would that make more sense to you?

Or like I do in Apache:Test/apache-test. That would be perhaps little bit better, but I am sure I do not see all the consequences. I will leave it up to your decision, I am fine with both possibilities. Feel free to make a new request with the comment included.
Comment 7 Martin Pluskal 2018-12-18 14:23:32 UTC
I would submit same changes to SLE-12 and SLE-15 maintenance if you dont mind - will check with maintenance coordinators about new subpackage.
Comment 8 Petr Gajdos 2018-12-18 14:26:52 UTC
Fine with me.
Comment 9 Swamp Workflow Management 2018-12-18 15:40:12 UTC
This is an autogenerated message for OBS integration:
This bug (1119396) was mentioned in
https://build.opensuse.org/request/show/659318 Factory / php7
Comment 11 Martin Pluskal 2019-02-12 09:38:31 UTC
Done
Comment 17 Swamp Workflow Management 2019-06-11 22:11:03 UTC
SUSE-SU-2019:1461-1: An update that solves 16 vulnerabilities and has two fixes is now available.

Category: security (moderate)
Bug References: 1118832,1119396,1126711,1126713,1126821,1126823,1126827,1127122,1128722,1128883,1128886,1128887,1128889,1128892,1129032,1132837,1132838,1134322
CVE References: CVE-2018-19935,CVE-2018-20783,CVE-2019-11034,CVE-2019-11035,CVE-2019-11036,CVE-2019-9020,CVE-2019-9021,CVE-2019-9022,CVE-2019-9023,CVE-2019-9024,CVE-2019-9637,CVE-2019-9638,CVE-2019-9639,CVE-2019-9640,CVE-2019-9641,CVE-2019-9675
Sources used:
SUSE Linux Enterprise Module for Web Scripting 15 (src):    php7-7.2.5-4.32.1
SUSE Linux Enterprise Module for Packagehub Subpackages 15 (src):    php7-7.2.5-4.32.1
SUSE Linux Enterprise Module for Open Buildservice Development Tools 15 (src):    php7-7.2.5-4.32.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 19 Swamp Workflow Management 2019-06-18 16:37:30 UTC
openSUSE-SU-2019:1572-1: An update that solves 16 vulnerabilities and has two fixes is now available.

Category: security (moderate)
Bug References: 1118832,1119396,1126711,1126713,1126821,1126823,1126827,1127122,1128722,1128883,1128886,1128887,1128889,1128892,1129032,1132837,1132838,1134322
CVE References: CVE-2018-19935,CVE-2018-20783,CVE-2019-11034,CVE-2019-11035,CVE-2019-11036,CVE-2019-9020,CVE-2019-9021,CVE-2019-9022,CVE-2019-9023,CVE-2019-9024,CVE-2019-9637,CVE-2019-9638,CVE-2019-9639,CVE-2019-9640,CVE-2019-9641,CVE-2019-9675
Sources used:
openSUSE Leap 15.1 (src):    php7-7.2.5-lp151.6.3.1
Comment 20 Swamp Workflow Management 2019-06-18 16:42:44 UTC
openSUSE-SU-2019:1573-1: An update that solves 16 vulnerabilities and has two fixes is now available.

Category: security (moderate)
Bug References: 1118832,1119396,1126711,1126713,1126821,1126823,1126827,1127122,1128722,1128883,1128886,1128887,1128889,1128892,1129032,1132837,1132838,1134322
CVE References: CVE-2018-19935,CVE-2018-20783,CVE-2019-11034,CVE-2019-11035,CVE-2019-11036,CVE-2019-9020,CVE-2019-9021,CVE-2019-9022,CVE-2019-9023,CVE-2019-9024,CVE-2019-9637,CVE-2019-9638,CVE-2019-9639,CVE-2019-9640,CVE-2019-9641,CVE-2019-9675
Sources used:
openSUSE Leap 15.0 (src):    php7-7.2.5-lp150.2.19.1
Comment 21 Swamp Workflow Management 2019-07-02 19:12:24 UTC
SUSE-SU-2019:1725-1: An update that solves two vulnerabilities and has one errata is now available.

Category: security (moderate)
Bug References: 1119396,1138172,1138173
CVE References: CVE-2019-11039,CVE-2019-11040
Sources used:
SUSE Linux Enterprise Software Development Kit 12-SP4 (src):    php7-7.0.7-50.80.2
SUSE Linux Enterprise Software Development Kit 12-SP3 (src):    php7-7.0.7-50.80.2
SUSE Linux Enterprise Module for Web Scripting 12 (src):    php7-7.0.7-50.80.2

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 23 Swamp Workflow Management 2019-10-24 10:12:27 UTC
SUSE-RU-2019:2760-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 1119396
CVE References: 
Sources used:
SUSE Linux Enterprise Module for Web Scripting 15-SP1 (src):    php7-7.2.5-4.43.2
SUSE Linux Enterprise Module for Web Scripting 15 (src):    php7-7.2.5-4.43.2
SUSE Linux Enterprise Module for Packagehub Subpackages 15 (src):    php7-7.2.5-4.43.2
SUSE Linux Enterprise Module for Open Buildservice Development Tools 15-SP1 (src):    php7-7.2.5-4.43.2
SUSE Linux Enterprise Module for Open Buildservice Development Tools 15 (src):    php7-7.2.5-4.43.2

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 24 Swamp Workflow Management 2019-10-31 11:14:27 UTC
openSUSE-RU-2019:2416-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 1119396
CVE References: 
Sources used:
openSUSE Leap 15.0 (src):    php7-7.2.5-lp150.2.28.1, php7-test-7.2.5-lp150.2.28.1
Comment 25 Swamp Workflow Management 2019-10-31 14:12:24 UTC
openSUSE-RU-2019:2417-1: An update that has one recommended fix can now be installed.

Category: recommended (moderate)
Bug References: 1119396
CVE References: 
Sources used:
openSUSE Leap 15.1 (src):    php7-7.2.5-lp151.6.12.1, php7-test-7.2.5-lp151.6.12.1