Creating Project Environments in Python with VSCode

---

Introduction

Creating Data Science projects can be pretty straightforward. With the numerous resources available these days, it’s just a matter of choosing a development tool and kicking off your project.

Documentation is also easily available, in addition to several AI bots that will help you working through mostly everything you want to create.

However, as the projects start to become more complex and professional, there will be a need to start isolating projects from one another. Sometimes, modules that work well together in project A, may fail to run together in project B. Or a method with the same name in two different packages can generate confusion. I mean, a lot can happen in non-isolated environments.

That is when we will find the need to start isolating Development environments. So, in this post, the idea is to show you a quick and easy way to create an isolated environment using Python and VS Code.

Let’s get to work.

Project Environments

As already mentioned, a development environment is an isolated "box" created inside your computer to install only the modules to be used for that project.

A development environment is an isolated "box" created inside your computer for better package control.

Imagine we will create a classification project that needs Pandas, Scikit Learn, and Streamlit. In this case, we can install only those modules and their dependencies, without the need to add many other packages that will never be used. It can then be separated from another project that won’t be using Streamlit, for example.

Now let’s move on and start coding a little.

Using Pip

The easiest way to create an environment is using the native tools from Python. To do that, just start a VS Code session and open a new Powershell terminal (Terminal > New Terminal).

Next, you can create a new folder for the project.

mkdir name_your_project

Then, change folders to access the recently created directory.

cd name_your_project

At this point, you can use VS Code to open the newly created folder, if you’d like. Just remember to re-open the Terminal from the new window.

Within the new folder, it is time to create the new environment. Use the following command. I will create a virtual environment using the standard name .venv .

python -m venv .venv

There you go. Now to activate it, you can use this command in Powershell.

.venv/Scripts/activate

Now, whatever you want to install when this environment is activated will be isolated under the test_proj and won’t affect the other projects. Let’s install Pandas and Streamlit, but not Scipy.

pip install pandas
pip install streamlit

Both packages are installed, if I run a quick script to check, here is the result.

import pandas as pd
import streamlit as st
print(pd.__version__)
print(st.__version__)

[OUT]:
2.2.2
1.38.0

If I check for Scipy import scipy .

import scipy
[OUT]: ModuleNotFoundError: No module named 'scipy'

If we create another environment now and install only Scipy, look what happens.

python -m venv env2
env2/Scripts/activate
pip install scipy

import pandas as pd
[OUT]: ModuleNotFoundError: No module named 'pandas'

See how Pandas is not installed in the env2. Let’s see about Scipy.

import scipy.stats as scs
print(scs.norm.rvs(loc=1, scale=3)

[OUT]:
0.5100109427428302

Works like a charm.

Using PyEnv and Poetry

Now let’s see another way to do the same thing with PyEnv and Poetry, two python packages suited for this purporse. This is even easier than the native tools from Python.

Using PyEnv is a good idea because it can manage different versions of Python in the same machine. When working with many projects, one problem that is always potential is that a given Python version is not compatible with a package that you are working with (or want to work with). In that case, you will need to install a previous or more updated version of Python. PyEnv comes to solve that.

Installing the package is a little tricky, but following this tutorial you can do that in no time for Windows.

Now to install Poetry, first you must install pipx. Follow these steps here. and then use the command pipx install poetry to finish installing Poetry.

You might also need to run this next command to tell your windows shell that you can let Poetry managing the virtual environment.

poetry config virtualenvs.in-project true

After the initial installation steps are done, to create a new environmento with Poetry is easy like this in your Powershell in VSCode:

poetry new test_proj  

Created package test_proj in test_proj

With that command, Poetry already creates the virtual environment folder, a test folder, and a nice pyproject.toml file with all the project specifications. It is amazing. Take a look at the toml file.

When I command poetry shell , Poetry creates the .venv folder and activates it.

Now, to add new packages to the project, you can use:

poetry add pandas

This gets added to the toml file:
[tool.poetry.dependencies]
Python = "^3.12"
pandas = "^2.2.2"

Or to remove them, use:

poetry remove pandas

To use a different version of Python for this project, we can ask PyEnv to use 3.11.5, for example.

pyenv local 3.11.5

Once done, you can write exit on the shell to deactivate your environment.

Before You Go

With that, we finish our little tour to understand a little more about environment management in Python using VS Code, Pip and Poetry.

This knowledge is useful to be able to isolate the effects of our projects in a controled "box", mitigating problems of dependencies or that famous "it worked on my machine".

I believe that the toml file generated by Poetry is also very useful and gives you a summary of what’s in the project. Additionally, Poetry does not bring all the dependencies listed. It displays only the packages you actually requested to install, like "Pandas", "Scipy", etc, instead of showing numpy, and other dependencies.

References

How to install pyenv on windows [2023]

Introduction

GitHub – pyenv/pyenv: Simple Python version management

Managing Multiple Python Versions With pyenv – Real Python