Bug 1095041 - python3-botocore: :'AWSHTTPSConnection' object has no attribute 'ssl_context'
python3-botocore: :'AWSHTTPSConnection' object has no attribute 'ssl_context'
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Development
Current
Other Other
: P3 - Medium : Normal (vote)
: ---
Assigned To: John Paul Adrian Glaubitz
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2018-05-29 13:29 UTC by Martin Wilck
Modified: 2022-02-25 21:05 UTC (History)
5 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Wilck 2018-05-29 13:29:08 UTC
Problem:

> mwilck@apollon:~> aws ec2 describe-instances
> 
> 'AWSHTTPSConnection' object has no attribute 'ssl_context'

See https://github.com/boto/botocore/issues/1258

According to 
https://github.com/boto/botocore/issues/1258#issuecomment-323469132,
this is caused by the packaged botocore replacing the vendor-supplied "requests" module with the official release.

I've verified this in OBS project home:mwilck:branches:devel:languages:python:aws.

Upgrading to the latest botocore version 1.10.28 does not fix the issue unless the code in the spec file that replaces the botocore-shipped requests module with the official requests module is removed.

Unfortunately, it's impossible to directly upgrade to the thus modified python3-botocore package, because rpm detects file conflicts caused by the %{$python_sitelib}/requests %{buildroot}%{$python_sitelib}/botocore/vendored/requests symlink in the current package.
Comment 1 Martin Wilck 2018-05-29 14:27:00 UTC
Upstream fixes to for botocore with native requests module:

https://github.com/boto/botocore/pull/1466
Comment 2 Martin Wilck 2018-05-29 14:30:23 UTC
So, in order to fix this problem, we could

 1) switch back to vendor-supplied requests module, as I did in my OBS project,
 2) Pull in the patches from comment 1 (not tested yet).

2) avoids the upgrade problem mentioned in comment 0, but requires pulling in a rather largeish patch set that hasn't been accepted upstream yet.
Comment 3 Tomáš Chvátal 2018-05-30 08:52:24 UTC
Bundling is not really good idea. I would really suggest to grab the upstream unbunding patch.

If you look at it and ommit the removals it is pretty tiny patchset to get it rolling.
Comment 4 Martin Wilck 2018-05-30 09:46:05 UTC
True, but the maintainers don't seem to embrace it (yet). That means that similar problems could occur again any time. 

So far we've just the patch series and a positive statement from a single guy on the github issue, probably a colleague of the patch author (both from Sweden).

OTOH, what's the actual problem with the bundling, other than general tidiness? The "vendored" part of botocore is ~3MiB, about 10% of the whole package, which is unlikely to be installed on minimal installations where size matters much, IMO.

As long as upstream (referring to botocore upstream) tests with the bundled packages, we may be better off with bundling them as well.
Comment 5 Tomáš Chvátal 2018-05-30 09:53:50 UTC
You plan to do the security audit and verification of those? Basically by default nothing that bundles stuff gets in factory and if it is included, mostly it happened by accident rather than by purpose.
Comment 6 Martin Wilck 2018-05-30 11:25:09 UTC
Sorry, I don't buy this. If you take this seriously, you need to do a security audit and verification of the whole botocore package, not only the bundled sub-packages (after all, the bundled subpackages are only for the use botocore and aws, nothing else).

I'm not aware that such auditing is part of package maintenance on openSUSE. We normally rely on the upstream authors to do the auditing, verification, and testing, don't we? And if we do, we should rather ship what upstream has verified.
Comment 7 Robert Schweikert 2018-05-30 13:19:50 UTC
The problem is that if a security issue is encountered in the bundled requests package then we are dependent on the maintainers of botocore to provide a patched requests implementation.

Given that getting things into boto and botocore is a rather lengthy undertaking the question is really; would you be OK if we'd expose your AWS credentials via a bug in the bundled requests package that we cannot fix in a hurry? 

My guess is that the answer to that question is No and thus having requests separate as it is today is the right thing to do.

We will pull the upstream patch to avoid the issue.
Comment 8 Martin Wilck 2018-05-30 14:07:22 UTC
Well, again, the argument can be extended to botocore as a whole. If we don't trust the maintainers to fix their security issues, we'd better not ship the package at all, maybe? (Note that a frequently encountered suggestion on the web is to simply pull awscli using pip, which currently of course comes down to using the bundled stuff as well - but I can see that in that case at least noone can blame SUSE).

But I'll stop arguing here. If you're going to pull the unbundling fixes from github, fine with me.
Comment 9 Martin Wilck 2018-06-06 07:12:23 UTC
Any progress? Has anyone gone for the backport of the un-vendoring patch set?
Comment 10 Martin Wilck 2018-08-28 09:23:45 UTC
Upstream has unvendored requests in 1.11.0.

https://github.com/boto/botocore/issues/1258#issuecomment-415928768

So this issue could be solved by updating python-botocore to 1.11.
Comment 11 John Paul Adrian Glaubitz 2018-08-28 09:25:22 UTC
(In reply to Martin Wilck from comment #10)
> Upstream has unvendored requests in 1.11.0.
> 
> https://github.com/boto/botocore/issues/1258#issuecomment-415928768
> 
> So this issue could be solved by updating python-botocore to 1.11.

I have an updated python-botocore package ready but it is blocked by python-urllib3 which is too old in Leap 42.3 and SLE-12-SP3 where I need to update it first.
Comment 14 Martin Wilck 2018-08-28 09:46:09 UTC
> See: https://build.opensuse.org/request/show/631815

yeah, saw that already.

Seriously: We are entering a dependency circle here.

The problem described in this bug isn't relevant for SLE-12 and Leap 42, as we still ship python-requests 2.11 there. This is still true for SLE-12-SP4, AFAICS. So, at least as far as this problem here is concerned, we can stay with botocore 1.10 just fine in SLE-12 and Leap 42.

OTOH, in factory, SLE15, and Leap15, we have the need to update botocore because it just doesn't work at all with the shipped python-requests (2.19 / 2.18, respectively).

So from my point of view it'd make an awful lot of sense to accept botocore 1.11 into factory and SLE15, while going with 1.10 for SLE12. I'm having a bit of a hard time trying to understand why SLE12 needs to get this cutting-edge update at all. Stuff like this ought to go to 15 in my understanding.
Comment 18 Swamp Workflow Management 2020-03-24 20:25:17 UTC
SUSE-RU-2020:0775-1: An update that has 11 recommended fixes can now be installed.

Category: recommended (important)
Bug References: 1069697,1075263,1088310,1095041,1118021,1118024,1118027,1129696,1136184,1146853,1146854
CVE References: 
Sources used:
SUSE OpenStack Cloud Crowbar 8 (src):    python-botocore-1.13.33-28.20.1, python-futures-3.0.2-15.3.1
SUSE OpenStack Cloud 8 (src):    python-botocore-1.13.33-28.20.1, python-futures-3.0.2-15.3.1
SUSE OpenStack Cloud 7 (src):    python-futures-3.0.2-15.3.1
SUSE Manager Tools 12 (src):    python-futures-3.0.2-15.3.1
SUSE Manager Server 3.2 (src):    python-futures-3.0.2-15.3.1
SUSE Manager Proxy 3.2 (src):    python-futures-3.0.2-15.3.1
SUSE Linux Enterprise Point of Sale 12-SP2 (src):    python-futures-3.0.2-15.3.1
SUSE Linux Enterprise Module for Public Cloud 12 (src):    python-boto3-1.10.33-14.14.1, python-botocore-1.13.33-28.20.1, python-futures-3.0.2-15.3.1, python-s3transfer-0.2.1-8.7.1
SUSE Linux Enterprise Module for Advanced Systems Management 12 (src):    python-futures-3.0.2-15.3.1
SUSE Enterprise Storage 5 (src):    python-futures-3.0.2-15.3.1
SUSE CaaS Platform 3.0 (src):    python-futures-3.0.2-15.3.1
HPE Helion Openstack 8 (src):    python-botocore-1.13.33-28.20.1, python-futures-3.0.2-15.3.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.
Comment 19 John Paul Adrian Glaubitz 2020-08-19 11:26:43 UTC
I can no longer reproduce this problem in Tumbleweed. Can this issue be closed?
Comment 20 Martin Wilck 2020-08-19 14:06:22 UTC
ok with me.
Comment 21 John Paul Adrian Glaubitz 2020-08-19 14:25:21 UTC
Fixed with updated packages.