If you are a linux geek and have always been compiling applications on linux and know readline library inside out, this article is probably not for you. I wrote this primarily because of lack of documentation on the problem that I was experiencing.
I was recently forced to compile postgres from source. I am using Amazon Linux version 1 that did not have binaries for postgresql 12.5. Usual stuff about pointing to yum repositories did not work. The only option I had was to compile postgresql from source. I followed the instructions and compiled it but you will see the setup complain about readline and zlib.
[ ~]$ ./configure
...
...
...
checking for library containing clock_gettime... none required
checking for library containing fdatasync... none required
checking for library containing sched_yield... none required
checking for library containing gethostbyname_r... none required
checking for library containing shmget... none required
checking for library containing readline... no
configure: error: readline library not found
If you have readline already installed, see config.log for details on the
failure. It is possible the compiler isn't looking in the proper directory.
Use --without-readline to disable readline support.
[ ~]$ ./configure --without-readline
...
...
...
checking for library containing dlopen... -ldl
checking for library containing socket... none required
checking for library containing shl_load... no
checking for library containing getopt_long... none required
checking for library containing crypt... -lcrypt
checking for library containing shm_open... -lrt
checking for library containing shm_unlink... none required
checking for library containing clock_gettime... none required
checking for library containing fdatasync... none required
checking for library containing sched_yield... none required
checking for library containing gethostbyname_r... none required
checking for library containing shmget... none required
checking for inflate in -lz... no
configure: error: zlib library not found
If you have zlib already installed, see config.log for details on the
failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.What is readline and zlib and why do we need it
Readline library provides support for programs that require line editing features. Readline includes autocomplete, moving the cursor etc. These are the features we take for granted. Morever, many programs need this kind of support and it makes sense to have a library so that the implementation is consistent across programs and users don’t have to deal with different ways of achieving same thing in different pograms. If you are writing a program that needs these capabilities, you need to include readline. Ex – in terminal, pressing ctrl+e to go to go to end of the command, ctrl+a to move to start of line. List of all capabilities are neatly listed on wikipedia and here. As you can see now, postgresql also uses readline to provide these features in psql. Sadly, I was facing trouble with auto complete and postgres documentation doesn’t list auto completion as a readline feature in its documentation where it talks about readline support. This can throw a person new to these things offguard and not provide a clear answer. I have submitted a request to update the documentation. Let’s see.
Zlib is used by postgresql in pg_dump and pg_restore to create compressed backups. You don’t want big backups that can otherwise be compressed. Read postgres documentation for more details.
Solution
Amazon Linux version 1 by default does not have development libraries for readline and zlib installed. Because the source files to use readline and zlib in postgresql are not installed by default. See the output below.
[ ~]$ sudo yum list installed | grep readline
readline.x86_64 6.2-9.14.amzn1 installed
[~]$ sudo yum list installed | grep zlib
zlib.x86_64 1.2.8-7.18.amzn1 installed
We will need to search for readline and zlib files needed for compiling postgresql. Search will tell us that the libraries we need are readline-devel.x86_64 and zlib-devel.x86_64.
[ ~]$ sudo yum search readline
Loaded plugins: priorities, update-motd, upgrade-helper
============================ N/S matched: readline =============================
compat-readline5-devel.x86_64 : Files needed to develop programs which use the
: readline library
compat-readline5-static.x86_64 : Static libraries for the readline library
libreadline-java-javadoc.x86_64 : Javadoc for libreadline-java
readline-devel.x86_64 : Files needed to develop programs which use the readline
: library
readline-static.x86_64 : Static libraries for the readline library
compat-readline5.i686 : A library for editing typed command lines
compat-readline5.x86_64 : A library for editing typed command lines
libreadline-java.x86_64 : Java wrapper for the EditLine library
perl-Term-UI.noarch : Term::ReadLine user interface made easy
readline.i686 : A library for editing typed command lines
readline.x86_64 : A library for editing typed command lines
[ ~]$ sudo yum search zlib
Loaded plugins: priorities, update-motd, upgrade-helper
======================================================= N/S matched: zlib =======================================================
jzlib.noarch : JZlib re-implementation of zlib in pure Java
jzlib-demo.noarch : Examples for jzlib
jzlib-javadoc.noarch : Javadoc for jzlib
perl-Compress-Raw-Zlib.x86_64 : Low-level interface to the zlib compression library
perl-IO-Zlib.noarch : Perl IO:: style interface to Compress::Zlib
zlib-devel.x86_64 : Header files and libraries for Zlib development
zlib-static.x86_64 : Static libraries for Zlib development
zlib.i686 : The compression and decompression library
zlib.x86_64 : The compression and decompression library
Next step would be to install readline and zlib header files and libraries.
[ ~]$ sudo yum install readline-devel.x86_64
...
skipping output for brevity
...
[ ~]$ sudo yum install zlib-devel.x86_64
...
skipping output for brevity
...Now the compilation will go on smoothly and without any issues. You will have psql with readline support and all the convenient features you wanted like autocomplete, history etc will work just fine.
Summary
- Amazon Linux version 1 does not have readline and zlib development libraries by default installed.
- Install them and then compile postgres instead of skipping them.
- You favorite psql features and compression when using pg_dump and pg_restore should work just fine.