Nowadays, there are so many stable ways to program with Python; you can use either use cloud services (such as colab or jupyter notebook) or you can simply install python globally into your local machine.

But, when you’re dealing with bunch of different Python projects, it is critical to be careful with the package dependencies. In order to address such issue, virtual environment is used and it boosts up the stability. Virtual environment allows you to create multiple separate Python environments into your local machine and this enables you to install separate Python packages (or libraries) per virtual environment.

As far as I know, there are two popular ways to manage virtual environment, which are anaconda (or so called conda) and Python’s venv. You can take a look at those two and choose as your preference. The venv which is a Python-provided way is capable of managing packages with light-weight but it cannot manage multiple environments with multiple Python versions. Since using the venv is much easier and simple than the conda, this post will be dealing with the venv only. For more information about how to setup a conda environment, take look at the this post.

Python’s venv

Fist off, let’s create a directory for virtual environments under home directory.

mkdir ~/venvs

Then you can create a new virtual environment into the ~/venvs directory by running the following command:

python -m venv ~/venvs/<your-venv>

And to activate the created virtual environment, run the following commands and verify whether it has been activated.

source ~/venvs/<your-venv>/bin/activate
which python

By running the which command, you will be able to see something like ~/venvs/<your-venv>/bin/python. If you are not able to see such result, make sure the venv is actually created.

You can simple deactivate the venv by running

deactivate

Installing Packages

Now that we create and activated the venv, we’re ready to install Python packages. Of course, you can simply run pip install but if the packages required for the project are too many, the dependencies between packages matter. There it is much easier to manage packages with file.

Create a simple text file (e.g. requirements.in) containing the packages you need as such:

numpy
pandas
pip-tools

I will be using pip-tools so run the following command to install pip-tools.

pip install pip-tools

Now, by using the pip-tools, you will be able to regenerate requirements.txt which contains each package’s version dependencies automatically. After that, you will be able to see that requirements.txt file is containing packages with versions. And, you can install the packages with ease by running the following!

pip-compile --output-file=requirements.txt requirements.in
pip install -r requirements.txt

If you would like to add new package(s) to your venv, you can add package(s) into requirements.in file and upgrade the dependencies.

pip-compile --output-file=requirements.dev.txt requirements.dev.in --upgrade
pip install -r requirements.txt