Debian readline/libedit breakage

[Joshua D. Drake]

Hello,

Per:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607109

It seems we may have a problem to consider. As far as I know, we are the
only major platform that supports libedit but our default is readline.
Unfortunately readline is not compatible with OpenSSL (apparently?)
licensing.

This seems that it may be a problem for us considering the pre-package
builds we do.

What does everyone think? Should we work on getting libedit up to snuff?

JD
--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579
Consulting, Training, Support, Custom Development, Engineering
http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Daniel Farina]

On Thu, Feb 10, 2011 at 2:34 PM, Joshua D. Drake <[hidden email]> wrote:

> Hello,
>
> Per:
>
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607109
>
> It seems we may have a problem to consider. As far as I know, we are the
> only major platform that supports libedit but our default is readline.
> Unfortunately readline is not compatible with OpenSSL (apparently?)
> licensing.
>
> This seems that it may be a problem for us considering the pre-package
> builds we do.
>
> What does everyone think? Should we work on getting libedit up to snuff?

I have to admit, this change in debian packaging -- which I have
noticed, and not a little -- makes my hands angry. I considered
looking into the problem, but were I doing it, I would have considered
teaching psql to support NSS or GnuTLS as totally viable alternatives
to this problem, as to keep readline.

--
fdr

--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Andrew Dunstan]

On 02/10/2011 05:34 PM, Joshua D. Drake wrote:

> Hello,
>
> Per:
>
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607109
>
> It seems we may have a problem to consider. As far as I know, we are the
> only major platform that supports libedit but our default is readline.
> Unfortunately readline is not compatible with OpenSSL (apparently?)
> licensing.
>
> This seems that it may be a problem for us considering the pre-package
> builds we do.
>
> What does everyone think? Should we work on getting libedit up to snuff?

I'll be happy if you do, but why haven't I haven't noticed, say, RedHat
taking this line?

cheers

andrew


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Tom Lane-2]

Andrew Dunstan <[hidden email]> writes:
> I'll be happy if you do, but why haven't I haven't noticed, say, RedHat
> taking this line?

Less narrow-minded interpretation of GPL requirements, perhaps.
(And yes, we have real lawyers on staff considering these issues.)

libedit is a long way from being ready to replace readline,
much as one could wish it otherwise.  If Debian want to shoot
themselves in the foot like that, we can't stop them, but neither
should we be devoting our project resources to fixing libedit.

(I have seen some noise recently on the Fedora lists about putting
work into libedit, so maybe something good will come of that.
I'm just not ready to define it as my/our problem.)

                        regards, tom lane

--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Alvaro Herrera-7]

Excerpts from Joshua D. Drake's message of jue feb 10 19:34:31 -0300 2011:
> Hello,
>
> Per:
>
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607109

O, the joy of having people mess up with legal stuff that nobody cares
about creating endless work for everyone.

--
Álvaro Herrera <[hidden email]>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Stephen Frost]

* Daniel Farina ([hidden email]) wrote:
> I have to admit, this change in debian packaging -- which I have
> noticed, and not a little -- makes my hands angry. I considered
> looking into the problem, but were I doing it, I would have considered
> teaching psql to support NSS or GnuTLS as totally viable alternatives
> to this problem, as to keep readline.

Supporting GnuTLS would be really nice..  That's how we addressed the
same issue w/ OpenLDAP (I was involved in that as a Debian
co-maintainer).  GnuTLS has limitations too, but in the end, I find
those more palatable (and the GnuTLS maintainer is certainly willing to
work on improving it) than dropping readline. :/

        THanks,

                Stephen

[Andrew Dunstan]

On 02/10/2011 06:36 PM, Stephen Frost wrote:

> * Daniel Farina ([hidden email]) wrote:
>> I have to admit, this change in debian packaging -- which I have
>> noticed, and not a little -- makes my hands angry. I considered
>> looking into the problem, but were I doing it, I would have considered
>> teaching psql to support NSS or GnuTLS as totally viable alternatives
>> to this problem, as to keep readline.
> Supporting GnuTLS would be really nice..  That's how we addressed the
> same issue w/ OpenLDAP (I was involved in that as a Debian
> co-maintainer).  GnuTLS has limitations too, but in the end, I find
> those more palatable (and the GnuTLS maintainer is certainly willing to
> work on improving it) than dropping readline. :/
>

Strikes me as a lot of work to buy nothing much.

cheers

andrew

--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Michael Banck]

On Thu, Feb 10, 2011 at 06:04:46PM -0500, Tom Lane wrote:
> Less narrow-minded interpretation of GPL requirements, perhaps.
> (And yes, we have real lawyers on staff considering these issues.)

Is their opinion public/can be made public?  This might possibly lead to
a re-evaluation of the situation by Debian.

> If Debian want to shoot themselves in the foot like that, we can't
> stop them

BTW, that change has been merged into Ubuntu and will be (as of now) in
the next Ubuntu release.


Michael

--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Greg Smith-21]

Michael Banck wrote:

On Thu, Feb 10, 2011 at 06:04:46PM -0500, Tom Lane wrote:
  

Less narrow-minded interpretation of GPL requirements, perhaps.
(And yes, we have real lawyers on staff considering these issues.)
    

Is their opinion public/can be made public?  This might possibly lead to
a re-evaluation of the situation by Debian.
  

I doubt that.  This is one of those situations where there is an ideological position held by the FSF and Debian that's unlikely to budge, but one that has limited testing in court.  I believe that RedHat's lawyers have assessed the business risk here and judged it not sufficient to worry about.  But their opinion on that isn't going to sway anyone evaluating this primarily on free software principles.

I had to trace down the history here once before while working on another project that couldn't link with readline; here's a timeline with all the latest fun parts at the end:

http://clisp.cvs.sourceforge.net/viewvc/clisp/clisp/doc/Why-CLISP-is-under-GPL : Early discussion with RMS about why linking with readline requires code using it be GPL, and why "the user did the linking" and "I wrapped it" aren't escape routes.

http://www.gnu.org/licenses/gpl-2.0.html : The GPLv2 includes the following in section 3:  "However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable."  This provides some relief to people building software on their own, but when you're the OS packager it doesn't help because you are providing the components and the executables.

http://lists.debian.org/debian-legal/2002/10/msg00113.html : Discussion of the exemption made for GPL libraries shipping with an OS, and an early mention of "Debian['s] current hardline position on the GPL+OpenSSL licensing issue"

http://bugs.ntp.org/show_bug.cgi?id=931 : ntp runs into readline concerns.  Pull quote:  "What is less clear is the claim that the FSF makes that any program written to even *use* the readline API is also a derived work. This hasn't been tested in court yet, so its validity is questionable. However, that is their claim."

http://people.gnome.org/~markmc/openssl-and-the-gpl.html : Why OpenSSL is particularly troublesome here.  This describes the now common "OpenSSL exemption" as a suggested workaround for projects who can modify their license terms.
 
http://lists.debian.org/debian-legal/2004/05/msg00595.html : Example of adding an OpenSSL exemption

http://archives.postgresql.org/pgsql-patches/2006-05/msg00040.php : A patch adding GnuTLS support is submitted to PostgreSQL.  It's rejected mainly because the code is so large/obtrusive.  TODO item "Consider GnuTLS if OpenSSL license becomes a problem" added.  [Hint:  it's now become a problem]

http://archives.postgresql.org/pgsql-hackers/2006-12/msg01213.php : More PostgreSQL discussion that predicts a collision with Debian policy is coming.  Concerns about the quality fo the GnuTLS API relative to the feature set provided by OpenSSL are raised too, as impediments toward switch away from OpenSSL.

http://archives.postgresql.org/pgsql-hackers/2006-12/msg01224.php : List of GPL applications that use libpq in Debian.

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=498857 : Python runs into the readline+OpenSSL issue

http://redmine.ruby-lang.org/issues/show/2982 : Ruby runs into the readline+OpenSSL issue

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=601754 : PostgreSQL runs into the readline+OpenSSL issue on Debian Lenny.  Note that this bug being open means it's possible all these problems in Squeeze are going to get backported to Lenny and break stable server installs all over the world one day in our near future.

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=603599 : Dupe of the bug for 9.0+Squeeze. This is the one that was "fixed" by switching to libedit.

Then we have the stream of bugs cascading out of that decision:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=605313 : Delete key stopped working in psql
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607109 : Cannot input multibyte characters in psql
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=607907 : Missing readline features
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=608442 : Input of non-ASCII characters broken
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=611918 : psql segfaults in libedit

So where are we at?

-GNU libreadine is certainly never going to add an OpenSSL exemption
-If the OpenSSL project was going to switch to a reasonable license, they'd have done it years ago
-There are many known and serious bugs/limitations in libedit relative to libreadline
-Adding GnuTLS support to PostgreSQL would require solving several code quality issues

Idealogically, I find the worst offendor here to be the OpenSSL license.  From a license purity perspective I'd like to see their ridiculous requirements bypassed altogether by doing whatever is necessary to get GnuTLS support working.  But pragmatically, fixing the bugs and adding features to libedit may be the easier route here.

[Stephen Frost]

* Greg Smith ([hidden email]) wrote:
> -GNU libreadine is certainly never going to add an OpenSSL exemption

I really wish they would, that's just them being obnoxious- it's already
LGPL, after all..

> -If the OpenSSL project was going to switch to a reasonable license,
> they'd have done it years ago

aiui, the problem here is actually a former OpenSSL hacker who has no
interest (and, in fact, a positive interest against) in changing the
OpenSSL licensing.  Most of the current OpenSSL hackers don't have an
issue with the change (again, aiui).

> -There are many known and serious bugs/limitations in libedit
> relative to libreadline

Yes, which makes it suck. :(

> -Adding GnuTLS support to PostgreSQL would require solving several
> code quality issues

I'm curious about this, but I don't know that I've got time to dive into
it and solve it. :/

> Idealogically, I find the worst offendor here to be the OpenSSL
> license.  From a license purity perspective I'd like to see their
> ridiculous requirements bypassed altogether by doing whatever is
> necessary to get GnuTLS support working.  But pragmatically, fixing
> the bugs and adding features to libedit may be the easier route
> here.

That suprises me..  There are a ton of tools which work with GnuTLS
today, and hearing that it's got serious issues isn't good. :/

        Thanks,

                Stephen

[Magnus Hagander-2]

On Fri, Feb 11, 2011 at 19:13, Stephen Frost <[hidden email]> wrote:
> * Greg Smith ([hidden email]) wrote:

>> -Adding GnuTLS support to PostgreSQL would require solving several
>> code quality issues
>
> I'm curious about this, but I don't know that I've got time to dive into
> it and solve it. :/

Yeah, I'm curious about that one as well. A lot has happened since
this was last investigated, I believe.

We may also have a problem in that libpq exposes OpenSSL structs,
though. We actually return it as a void *, to make it possible to
change, but there's no API in libpq to tell you what it is...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Stephen Frost]

* Magnus Hagander ([hidden email]) wrote:
> We may also have a problem in that libpq exposes OpenSSL structs,
> though. We actually return it as a void *, to make it possible to
> change, but there's no API in libpq to tell you what it is...

Ugh, yeah, that probably wasn't the best decision in the world.. :(

        Stephen

[Joshua D. Drake]

On Fri, 2011-02-11 at 14:59 +0100, Michael Banck wrote:
> On Thu, Feb 10, 2011 at 06:04:46PM -0500, Tom Lane wrote:
> > Less narrow-minded interpretation of GPL requirements, perhaps.
> > (And yes, we have real lawyers on staff considering these issues.)
>
> Is their opinion public/can be made public?  This might possibly lead to
> a re-evaluation of the situation by Debian.

I certainly hope so. Although, what I question is... Did Debian seek
legal advice? Debian does have a corporation of which I am a director
for. Software in the Public Interest. I don't recall a legal request
coming through from the DPL?

>
> > If Debian want to shoot themselves in the foot like that, we can't
> > stop them
>
> BTW, that change has been merged into Ubuntu and will be (as of now) in
> the next Ubuntu release.

Yeah see, that is something that raises my red-alert bells. As popular
as Debian is, the "user" population is squarely in Ubuntu world and that
has some serious public implications as a whole.

Sincerely,

Joshua D. Drake

--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579
Consulting, Training, Support, Custom Development, Engineering
http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Greg Smith-21]

Stephen Frost wrote:

-Adding GnuTLS support to PostgreSQL would require solving several
code quality issues
    

I'm curious about this, but I don't know that I've got time to dive into
it and solve it. :/
  

Note that the past discussion was on the difficulty of matching the existing OpenSSL API using GnuTLS, which is apparently difficult to do.  I wasn't trying to suggest there were issues specificially with GnuTLS's code quality.  It's more that the APIs are just different enough that it's not trivial to do a swap--which is surprising given how many people have seemingly needed to do exactly this conversion.  You'd think there'd be a simple "OpenSSL-like" interface available for GnuTLS by now or something.

[Magnus Hagander-2]

On Fri, Feb 11, 2011 at 20:09, Greg Smith <[hidden email]> wrote:

> Stephen Frost wrote:
>
> -Adding GnuTLS support to PostgreSQL would require solving several
> code quality issues
>
>
> I'm curious about this, but I don't know that I've got time to dive into
> it and solve it. :/
>
>
> Note that the past discussion was on the difficulty of matching the existing
> OpenSSL API using GnuTLS, which is apparently difficult to do.  I wasn't
> trying to suggest there were issues specificially with GnuTLS's code
> quality.  It's more that the APIs are just different enough that it's not
> trivial to do a swap--which is surprising given how many people have
> seemingly needed to do exactly this conversion.  You'd think there'd be a
> simple "OpenSSL-like" interface available for GnuTLS by now or something.

There is one, but it's not complete - it will work for simple users,
though, AFAIK.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Robert Haas]

On Fri, Feb 11, 2011 at 2:09 PM, Greg Smith <[hidden email]> wrote:
> Note that the past discussion was on the difficulty of matching the existing
> OpenSSL API using GnuTLS, which is apparently difficult to do.

I believe that the OpenSSL API is "make some function calls, and if it
works, then you're using it right; if not, copy some source code from
the examples and use the undocumented APIs that appear there to fix
whatever problem you're having."

At least, that's been my approach.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Alvaro Herrera-7]

Excerpts from Greg Smith's message of vie feb 11 14:51:17 -0300 2011:

> So where are we at?
>
> -GNU libreadine is certainly never going to add an OpenSSL exemption
> -If the OpenSSL project was going to switch to a reasonable license,
> they'd have done it years ago
> -There are many known and serious bugs/limitations in libedit relative
> to libreadline
> -Adding GnuTLS support to PostgreSQL would require solving several code
> quality issues

Why do we have to involve the whole of PostgreSQL?  Since the only piece
that links to libreadline is psql, perhaps we could fix this by having
only psql optionally use GnuTLS.  (I don't know if you can make an
OpenSSL server talk to a GnuTLS client -- are these things supposed to
be interoperable?)

--
Álvaro Herrera <[hidden email]>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Daniel Farina-4]

On Fri, Feb 11, 2011 at 11:49 AM, Alvaro Herrera
<[hidden email]> wrote:
> Why do we have to involve the whole of PostgreSQL?  Since the only piece
> that links to libreadline is psql, perhaps we could fix this by having
> only psql optionally use GnuTLS.  (I don't know if you can make an
> OpenSSL server talk to a GnuTLS client -- are these things supposed to
> be interoperable?)

I agree with this: barring shockingly convenient engineering details,
my plan was to just evaluate the option of doing this for the psql
client.

--
fdr

--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Charles.McDevitt]

Don't forget that OpenSSL has a FIPS-140 compliant version, and FIPS-140 compliance is essential to many Federal users.

GnuTLS doesn't qualify.



--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[Charles.McDevitt]

If psql uses libreadline and libgnutls, does that mean psql will be distributed under the GPL in the future?  Or Dual-licensed?

If I read the readline license right, applications that link to it must be GPL.

That's why we (EMC/Greenplum) switch to libedit, even though readline is nicer... We didn't want to ship part of our product as GPL


--
Sent via pgsql-hackers mailing list ([hidden email])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers