Installing Psycopg2 on Virtualenv

Hi, I am trying to install psycopg2 on one of my virtual environments. I have installed postgre server and my machine is Ubuntu 16.04. psql works and I have installed psycopg2 on my main python path. However, I usually work on virtual environments since I do not want to play with system python. I did google it; but, I got a couple of old blog posts. None of them worked for me.

Has anyone installed psycopg2 on virtualenv? Or is there anyone can help me?

Many thanks,

Tumurtogtokh

What is the difference between installing psycopg2 on main path and venv?!
You need to create / activate your desired virtual environment with:

$ cd project_folder
$ virtualenv venv
$ source venv/bin/activate
$ pip install psycopg2

If you still get an error, try this before repeating the steps above:

$ sudo apt update
$ sudo apt install libpq-dev python-dev

Hope this may help.

Thanks for your reply. Basically, I do not want to install python packages on my main python as it may cause some problem. So I always use virtualenv for development. I use my virtualenv activated.

I don’t know why psycopg2 see it is as different though. I have followed your advice and still got an error I have received before. The error is:

Building wheels for collected packages: psycopg2
Building wheel for psycopg2 (setup.py) … error
ERROR: Command errored out with exit status 1:
command: /home/tumurtogtokh/.virtualenvs/SE/bin/python3.6 -u -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’"’/tmp/pip-install-jh5js8l3/psycopg2/setup.py’"’"’; file=’"’"’/tmp/pip-install-jh5js8l3/psycopg2/setup.py’"’"’;f=getattr(tokenize, ‘"’"‘open’"’"’, open)(file);code=f.read().replace(’"’"’\r\n’"’"’, ‘"’"’\n’"’"’);f.close();exec(compile(code, file, ‘"’"‘exec’"’"’))’ bdist_wheel -d /tmp/pip-wheel-4gmajjae --python-tag cp36
cwd: /tmp/pip-install-jh5js8l3/psycopg2/
Complete output (38 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/psycopg2
copying lib/tz.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/_json.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/extensions.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/pool.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/errors.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/_range.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/sql.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/extras.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/compat.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/init.py -> build/lib.linux-x86_64-3.6/psycopg2
copying lib/errorcodes.py -> build/lib.linux-x86_64-3.6/psycopg2
running build_ext
building ‘psycopg2._psycopg’ extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/psycopg
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.8.4 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=110005 -DHAVE_LO64=1 -I/usr/include/python3.6m -I/home/tumurtogtokh/.virtualenvs/SE/include/python3.6m -I. -I/usr/include/postgresql -I/usr/include/postgresql/11/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.6/psycopg/psycopgmodule.o -Wdeclaration-after-statement
In file included from psycopg/psycopgmodule.c:27:0:
./psycopg/psycopg.h:34:20: fatal error: Python.h: No such file or directory
compilation terminated.

It appears you are missing some prerequisite to build the package from source.

You may install a binary package by installing ‘psycopg2-binary’ from PyPI.
If you want to install psycopg2 from source, please install the packages
required for the build and try again.

For further information please check the ‘doc/src/install.rst’ file (also at
http://initd.org/psycopg/docs/install.html).

error: command ‘x86_64-linux-gnu-gcc’ failed with exit status 1

ERROR: Failed building wheel for psycopg2

It seems it is complaining about some prerequisite installation. Since, psycopg2 is installed on my main python, I do believe I have all prerequisite installed. Here is pip list of my main python:

Package Version


adium-theme-ubuntu 0.3.4
asn1crypto 0.24.0
cffi 1.12.3
configparser 4.0.2
cryptography 2.7
entrypoints 0.3
enum34 1.1.6
ipaddress 1.0.22
keyring 18.0.1
mpi4py 3.0.2
pbr 5.4.3
pip 19.2.3
psycopg2 2.7.6.1
pycparser 2.19
SecretStorage 2.3.1
setuptools 20.7.0
six 1.12.0
stevedore 1.31.0
unity-lens-photos 1.0
virtualenv 16.7.5
virtualenv-clone 0.5.3
virtualenvwrapper 4.8.4
wheel 0.29.0

Many thanks,

Tumurtogtokh

Those prerequisites must be installed in virtualenv as well since this is isolated Python installation.
pip list will show you all packages installed in your virtual environment.

Also, please, check the version of Python. virtualenv is for Python 2. If you need to create virtual environment using Python 3, then use this command:
$ python3 -m venv project-env

I’ll if there any other things to consider and will come back later.

Those packages are installed by cffi, mpi4py. Otherwise, I did not install any package. Maybe psycopg2 installed some of its prerequisite. And my main python is 2 as it is in ubuntu and my virtualenv python is 3. As I use virtualenvwrapper, virtualenv is managed properly. Before trying to install psycopg2 , I installed postgresql-11 (sudo apt-get install postgresql-11) following a guide from https://www.postgresql.org/download/linux/ubuntu/.

Any ideas to fix this issue?

Many thanks,

Tumurtogtokh

Hi @tumurtogtokh

I would recommend you to use Anaconda for these purposes. It provides the same possibilities but it is much more stable in my opinion.

Once you have Anaconda you can create a virtual environment using:

conda create --name my_environment_name

Then to activate it you do:

conda activate my_environment_name

On there you can install whatever you want without affecting the rest.

Full example for psycopg2:

conda create --name postgres
conda activate postgres
conda install -c anaconda psycopg2 

If you really want to use python virtual environments let me know and I will try to figure out what is going on.

Hi @Francois,

Thank you for your reply. With Anaconda it worked. If you don’t mind, it would be nice to know what is going on with python virtual environment.

Many thanks,

Tumurtogtokh