Bug 1011582

Submitted to 42.2 from openSUSE:Factory by Michael.

(In reply to Falk Schönfeld from comment #0)
> Nov 22 11:35:08 zwinger slapd[4582]: config error processing
> olcDatabase={1}hdb,cn=config:
> Nov 22 11:35:08 zwinger slapd[4582]: DIGEST-MD5 common mech free

Falk, please attach configuration.

What command should I use to dump configuration?

zwinger:/var/lib/ldap # cat /etc/openldap/slapd.d/cn\=config/olcDatabase\=\{0\}config.ldif 
# AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify.
# CRC32 1ea7a0d9
dn: olcDatabase={0}config
objectClass: olcDatabaseConfig
olcDatabase: {0}config
olcRootDN: cn=config
structuralObjectClass: olcDatabaseConfig
entryUUID: 7f1f2ea4-7633-1035-9716-f9b440bd1ee0
creatorsName: cn=config
createTimestamp: 20160304090158Z
olcRootPW:: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
olcSecurity: simple_bind=128 ssf=71
entryCSN: 20160304091127.769845Z#000000#000#000000
modifiersName: cn=config
modifyTimestamp: 201603040
zwinger:/var/lib/ldap #

zwinger:/var/lib/ldap #  slapcat -b cn=config
583436b3 UNKNOWN attributeDescription "OLCDBCACHESIZE" inserted.
583436b3 UNKNOWN attributeDescription "OLCDBCHECKPOINT" inserted.
583436b3 UNKNOWN attributeDescription "OLCDBCONFIG" inserted.
583436b3 UNKNOWN attributeDescription "OLCDBIDLCACHESIZE" inserted.
583436b3 UNKNOWN attributeDescription "OLCDBINDEX" inserted.
583436b3 config error processing olcDatabase={1}hdb,cn=config: 
slapcat: bad configuration file!
zwinger:/var/lib/ldap #

probably the same as #971669

To be more clear about the reason for this inconvenience:
The database backends are now compiled as dynamically loadable modules.
You have to tweak your configuration to actually load back-hdb.

In entry cn=module{0},cn=config you need

olcModuleLoad: {0}back_hdb

and more lines for any formerly static backend or overlay module.

Can you still export your config with slapcat -n0?

zwinger:/var/lib/ldap # slapcat -n0
58346e5b UNKNOWN attributeDescription "OLCDBCACHESIZE" inserted.
58346e5b UNKNOWN attributeDescription "OLCDBCHECKPOINT" inserted.
58346e5b UNKNOWN attributeDescription "OLCDBCONFIG" inserted.
58346e5b UNKNOWN attributeDescription "OLCDBIDLCACHESIZE" inserted.
58346e5b UNKNOWN attributeDescription "OLCDBINDEX" inserted.
58346e5b config error processing olcDatabase={1}hdb,cn=config: 
slapcat: bad configuration file!
zwinger:/var/lib/ldap #

right, after adding 

olcModuleLoad: {0}back_hdb

in cn=config/olcDatabase={0}config.ldif

slapd is starting.

Just ran into the same issue which was a very annoying experience given that upgrading in one Leap major version should be rather painless.

It also was quite horrible to find a solution if not working with openldap's cn=config everyday. I finally made it work by 
- removing the olcDatabase={1}hdb.ldif from cn=config
- starting slapd
- rerun yast module which added cn=module{0}.ldif
- copy back olcDatabase={1}hdb.ldif

I'm not saying that thos upgraded must not happen in minor Leap upgrades but they certainly need to be well documented!!!

Shouldn't it be added to https://doc.opensuse.org/release-notes/x86_64/openSUSE/Leap/42.2/ for example?

Is there any way to mitigate that upgrade issue?
Either by a patch for LDAP in 42.2?
Or an official upgrade note how to do it right?

Is there any way to fix the config automatically or patch openldap to load the right modules automatically?

If not please add a paragraph to the release notes

Hey Michael, do you see an opportunity to patch both slapd.conf and olc configuration in an RPM upgrade to automatically load the default DB drivers?

If not, I'll coordinate with documentation colleagues to note down the manual way in release notes.

to clarify - I was wonder whether there is a scripted approach to modify existing slapd.conf and olc configurations to load default DB modules. The script (if any) does not have to be an RPM post-install script.

Sorry for the late reply. Time constraints.

Actually this is caused by a combination of two things:

1. dynamic slapd configuration without attribute olcModuleLoad in cn=module{0},cn=config, probably generated by yast configuration
Probably it'd be a good idea to also fix this in yast.
 
2. the removal of statically linked backends/overlays which I proposed to have OpenLDAP for memory-constrained systems (rpi1) and meet security compliance (remove dead code)
see openldap2.changes:
"- All backends and overlays as modules for smaller memory footprint
   on memory constrained systems"

One could re-enable static linking of some modules (e.g. back-hdb):
https://build.opensuse.org/package/rdiff/home:stroeder:branches:network:ldap/openldap2?opackage=openldap2&oproject=network%3Aldap

But in the long run users should be strongly encouraged to migrate to back-mdb.
So personally I'd prefer the above only for Leap.

Another useful safety net if anything fails during upgrade would be to make backup of running config in pre-installation:
slapcat -b cn=config -l /var/lib/slapd-config-<timestamp>.ldif

Thanks Michael.

I'll note down the issue with Yast.

I like the idea of separating database drivers from main executable, configuration wise user should be advised of the change and adjust slapd.conf or cn=config accordingly. I'll compose a piece of text for release note and let you take a look.

How about:

OpenLDAP database drivers
=========================
Prior to Leap 42.2, the OpenLDAP main executable contains commonly used database drivers such as "hdb" and "bdb". In 42.2, the database drivers are seperated from main executable in an effort to reduce memory footprint and eliminate runtime dead code. All database drivers are now loaded explicitly, as instructed by configuration file "slapd.conf" or online configuration "cn=config".

If you are using database drivers previously built into OpenLDAP executable, namely "bdb", "hdb", and "mdb", then you must alter OpenLDAP configuration to explicitly load database driver, or OpenLDAP server will fail to start.

If you are currently using "slapd.conf" configuration file, you may add these lines into the file:

    modulepath /usr/lib64/openldap
    moduleload back_bdb.la
    moduleload back_hdb.la
    moduleload back_mdb.la

You may omit one or more "moduleload" lines among the three, if you can determine the type of drivers your existing LDAP databases use.

If you are currently using online configuration "cn=config", use "ldapadd" command to add the following entries into "cn=config" database:

# ldapadd -Y EXTERNAL -H ldapi:///
dn: cn=module{0},cn=config
objectClass: olcModuleList
cn: module{0}
olcModulePath: /usr/lib64/openldap
olcModuleLoad: back_bdb.la
olcModuleLoad: back_hdb.la
olcModuleLoad: back_mdb.la

Note to self: write down that section of text into SLE 15 release notes as well.

@Howard, may I ask where the above was added to the documentation? Or are you waiting on confirmation from anyone still?

I just upgraded my next Leap and tried to apply this.

(In reply to Howard Guo from comment #14)
> If you are currently using online configuration "cn=config", use "ldapadd"
> command to add the following entries into "cn=config" database:
> 
> # ldapadd -Y EXTERNAL -H ldapi:///
> dn: cn=module{0},cn=config
> objectClass: olcModuleList
> cn: module{0}
> olcModulePath: /usr/lib64/openldap
> olcModuleLoad: back_bdb.la
> olcModuleLoad: back_hdb.la
> olcModuleLoad: back_mdb.la

ldapadd does not work because slapd does not even start. So this documentation does not seem to be complete.
So again I 
- removed olcDatabase={1}mdb.ldif from cn=config
- started slapd
- executed the above ldapadd
- restarted slapd
-> still no LDAP database loaded
- copied back olcDatabase={1}mdb.ldif
- restarted slapd
-> thankfully back in operation

It would really be nice if that would be documented at some point and also in a way that it works.
Unfortunately I'm absolutely no expert in that area and still don't know what the "right way" is!

Thanks for the feedback Wolfgang.

Michael: openldap does not officially recommend manually manipulating slapd's cn=config directory on disk, though I always find it to be easy and painless, I guess it's similar to situation with "vi /etc/sudoers" vs visudo. Do you see an opportunity to fix cn=config without having to manipulate cn=config directory on disk?

Ideally the ldapadd command posted in Comment 14 should be used prior running the upgrade. Could someone with more RPM skills add such a auto-magic command if there is an existent setup with cn=config without these modules being loaded?

But anyhow something could go wrong in between. That's why I already recommended in Comment 12 to invoke this command *before* the upgrade:

slapcat -b cn=config -l /var/lib/slapd-config-<timestamp>.ldif

The last few days I've made upgrade from 42.1 (working) to 42.3
Hopefully this bug was opened, and I can refer to it to readapt the configuration.

This is what I've discovered the proposed command is not loading enough module to make slapd happy when syncronisation is in place.

I've extended to

    ldapadd -Y EXTERNAL -H ldapi:///
dn: cn=module{0},cn=config
objectClass: olcModuleList
cn: module{0}
olcModulePath: /usr/lib64/openldap
olcModuleLoad: back_bdb.la
olcModuleLoad: back_hdb.la
olcModuleLoad: back_mdb.la
olcModuleLoad: back_relay.la
olcModuleLoad: back_meta.la
olcModuleLoad: syncprov.la

perhaps back_relay and back_meta are not mandatory, but I guess it is better to have more available than not enough.

About the packaging skill (The neededinfo is already too older as we miss the fix for Leap 42.3, strangely this bug was not tagged as Important for release)
and not referenced in the 42.3 release note.

https://en.opensuse.org/openSUSE:Packaging_scriptlet_snippets

I've added jengelh in cc (one reference in terms of packaging too ;-)

*** Bug 964924 has been marked as a duplicate of this bug. ***

This is automated batch bugzilla cleanup.

The openSUSE 42.2 changed to end-of-life (EOL [1]) status. As such
it is no longer maintained, which means that it will not receive any
further security or bug fix updates.
As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
openSUSE, or you can still observe it under openSUSE Leap 15.0, please
feel free to reopen this bug against that version (see the "Version"
component in the bug fields), or alternatively open
a new ticket.

Thank you for reporting this bug and we are sorry it could not be fixed
during the lifetime of the release.

[1] https://en.opensuse.org/Lifetime