Question

Generate requirements.txt from Python Imports Automatically

pythondependenciespython-importrequirements.txt

Question

Given a directory containing Python source code downloaded from GitHub, can a requirements.txt file be generated automatically by examining the project's import statements? How reliable is this approach when the repository does not already provide dependency information?

Short Answer

You can generate a useful starting requirements.txt by scanning Python imports, usually with a tool such as pipreqs. However, imports do not always reveal the exact package name or version required, so generated files should be reviewed and tested in a clean virtual environment.

Concept

A requirements.txt file lists the distributions that pip must install for a project. For example:

requests==2.32.3
flask>=3.0

Python code, on the other hand, imports modules:

import requests
from flask import Flask

These names often match, but they are not guaranteed to match. A dependency-scanning tool reads imports in .py files, ignores standard-library modules where possible, and maps import names to installable PyPI package names.

This is helpful when a repository has no installation instructions, but it cannot perfectly reconstruct the author's environment:

  • An imported module may come from a package with a different name.
  • A dependency may be loaded dynamically and never appear in an import statement.
  • Optional features may require extra packages.
  • The code may depend on a particular version that imports alone cannot reveal.
  • Imports can be local modules within the repository, not third-party packages.

Therefore, import scanning creates a candidate dependency list, not a guarantee that the application will run.

Mental Model

Think of imports as a recipe's ingredient names and requirements.txt as a shopping list.

If the recipe says import requests, buying requests is straightforward. But sometimes the ingredient label does not match the store's product name. For example, code imports yaml, while the package installed with pip is named PyYAML.

A scanner can read the recipe and build most of the shopping list, but you still need to cook the recipe once to discover missing, optional, or version-specific ingredients.

Take Quiz

Syntax and Examples

The most common tool for generating requirements from source imports is pipreqs.

Install it into the environment you use for development:

python -m pip install pipreqs

Run it from the parent directory of the project:

pipreqs ./my_project

This writes my_project/requirements.txt by default. To replace an existing file, use:

pipreqs ./my_project --force

For example, given this file:

# app.py
import requests
from flask import Flask
import json

app = Flask(__name__)

A scanner should include packages similar to:

Flask==3.0.0
requests==2.32.3

It should not include json, because json belongs to Python's standard library and does not need to be installed separately.

After generation, install the result in a fresh environment:

python -m pip install -r requirements.txt

Then run the project's tests or application entry point.

Step by Step Execution

Consider this project:

# report.py
import csv
import pandas as pd
from dateutil import parser

When an import scanner processes the file:

  1. It finds csv.
  2. It recognizes csv as part of the Python standard library, so it does not add it to requirements.txt.
  3. It finds pandas and looks for the distribution that provides it. The distribution is also named pandas.
  4. It finds the dateutil module. The installable distribution is commonly named python-dateutil, not dateutil.
  5. It writes the discovered third-party packages, often with versions obtained from package metadata or an index.

The output may look like this:

pandas==2.2.2
python-dateutil==2.9.0.post0

The alias pd does not matter: import pandas as pd still identifies pandas as the imported module.

Real World Use Cases

  • Downloaded example projects: Create an initial dependency list before trying to run a GitHub repository that lacks setup instructions.
  • Legacy internal scripts: Document dependencies for an old script whose original virtual environment was lost.
  • Small automation tools: Scan a collection of Python scripts before packaging or sharing them with teammates.
  • Migration work: Discover likely dependencies while moving a script into a maintained project with a virtual environment and tests.
  • Code review and cleanup: Compare scanned imports with an existing dependency file to find packages that may no longer be used.

For a published library or production application, the preferred source of dependency information is a maintained project configuration file such as pyproject.toml, not a scanner-generated file.

Take Quiz

Real Codebase Usage

In maintained projects, developers usually declare dependencies deliberately in pyproject.toml and create a locked environment from that declaration. Import scanning is most useful for recovery or auditing.

A practical recovery workflow is:

  1. Create a virtual environment.
  2. Run pipreqs to create an initial list.
  3. Install the list.
  4. Run tests, a CLI command, or the application.
  5. Add packages reported by real runtime errors.
  6. Record version constraints after confirming compatible versions.

Use imports carefully in code as well. Imports at the top of a file make dependencies easier for tools to discover:

import requests


def fetch_status(url: str) -> int:
    response = requests.get(url, timeout=10)
    response.raise_for_status()
    return response.status_code

A dynamic import is harder for scanners to detect:

import importlib

plugin = importlib.import_module("my_optional_plugin")

In a real codebase, document optional plugins explicitly, for example in pyproject.toml optional dependencies or project documentation.

Common Mistakes

Treating generated output as complete

A generated file can miss dynamic imports, optional features, test dependencies, and packages required only in certain operating systems.

Avoid it: install into a clean environment and run the project.

Using pip freeze as if it scans source code

python -m pip freeze > requirements.txt

This command records everything installed in the current environment, including unrelated tools. It does not inspect the project imports.

Avoid it: use pipreqs for an import-based starting point; use pip freeze only when the environment is known to be dedicated to that project.

Installing standard-library modules

This is unnecessary and may fail:

json
os
pathlib

Avoid it: standard-library imports do not belong in requirements.txt.

Assuming import names always equal package names

This is incorrect:

yaml

The common package name is:

PyYAML

Other examples include PIL from Pillow and sklearn from scikit-learn.

Avoid it: review unusual module names and verify them with project documentation or PyPI.

Overwriting a hand-maintained file without checking it

pipreqs . --force

This can discard version constraints or manually documented optional dependencies.

Avoid it: back up or compare the existing file before replacing it.

Comparisons

ApproachWhat it usesBest forMain limitation
pipreqsPython import statementsRecovering likely direct dependencies from sourceCan miss dynamic, optional, and non-imported runtime dependencies
pip freezePackages installed in the current environmentReproducing a known, isolated environmentOften includes unrelated and transitive packages
pyproject.tomlDependencies declared by maintainersNew and actively maintained projectsMust be kept up to date manually
requirements.txt written by handIntentional package choices and constraintsSmall scripts and deployment requirementsEasy to forget a dependency without tests

pipreqs answers “what does this source appear to import?” pip freeze answers “what is installed right now?” These are different questions.

Take Quiz

Cheat Sheet

# Create and activate an isolated environment
python -m venv .venv

# macOS/Linux
source .venv/bin/activate

# Windows PowerShell
.venv\Scripts\Activate.ps1

# Install import scanner
python -m pip install pipreqs

# Generate requirements.txt for a directory
pipreqs ./my_project

# Replace an existing generated file
pipreqs ./my_project --force

# Install the generated requirements
python -m pip install -r ./my_project/requirements.txt

Rules to remember:

  • Imports identify modules; pip installs distributions.
  • Standard-library modules such as os, json, and pathlib are not dependencies.
  • Check package-name mismatches such as yaml → PyYAML.
  • Test in a clean virtual environment.
  • Prefer a maintained pyproject.toml when you control the project.

FAQ

Can Python create requirements.txt directly from imports?

Python's standard library does not provide a complete built-in command for this. Third-party tools such as pipreqs scan imports and generate a useful file.

Is pipreqs always accurate?

No. It is a best-effort scanner. It may miss dynamically imported packages, optional dependencies, version requirements, and some module-to-package name mappings.

What is the difference between pipreqs and pip freeze?

pipreqs scans project source files. pip freeze lists packages already installed in the active environment.

Why does pipreqs generate a package name different from my import?

Python module names and PyPI distribution names can differ. For example, import yaml normally requires installing PyYAML.

Should I include test packages in requirements.txt?

Include them only if the file is intended for development and testing. Many projects keep runtime and development dependencies separate.

How do I verify a generated requirements file?

Create a new virtual environment, install with python -m pip install -r requirements.txt, and run tests or the application.

What if the repository has pyproject.toml but no requirements.txt?

Use the dependency configuration in pyproject.toml. It is usually the project's intended dependency declaration and is more reliable than import scanning.

Take Quiz

Mini Project

Description

Create a dependency-recovery workflow for a downloaded Python project. You will scan its imports, install the generated dependencies in an isolated environment, and verify that the project starts successfully.

Goal

Generate and validate a requirements.txt file without polluting your global Python installation.

Requirements

Create a virtual environment in or beside the downloaded project directory. Install pipreqs in the active virtual environment. Generate a requirements.txt file by scanning the project source. Install the generated dependencies into the same virtual environment. Run an available test command, script, or application entry point and record any missing dependency.

Take Quiz

Keep learning

Related questions

View more