Understanding PyInstaller Hooks
Note
We strongly encourage package developers to provide hooks with their packages. See section Providing PyInstaller Hooks with your Package for how easy this is.
In summary, a “hook” file extends PyInstaller to adapt it to the special needs and methods used by a Python package. The word “hook” is used for two kinds of files. A runtime hook helps the bootloader to launch an app. For more on runtime hooks, see Changing Runtime Behavior. Other hooks run while an app is being analyzed. They help the Analysis phase find needed files.
The majority of Python packages use normal methods of importing their dependencies, and PyInstaller locates all their files without difficulty. But some packages make unusual uses of the Python import mechanism, or make clever changes to the import system at runtime. For this or other reasons, PyInstaller cannot reliably find all the needed files, or may include too many files. A hook can tell about additional source files or data files to import, or files not to import.
A hook file is a Python script, and can use all Python features.
It can also import helper methods from PyInstaller.utils.hooks
and useful variables from PyInstaller.compat
.
These helpers are documented below.
The name of a hook file is hook-full.import.name.py
,
where full.import.name is
the fully-qualified name of an imported module.
For example, hook-PyQt5.QtCore.py
is a hook file corresponding to
the module PyQt5.QtCore
. When your script (or one of its dependencies)
contains import PyQt5.QtCore
(or from PyQt5 import QtCore
),
Analysis notes that hook-PyQt5.QtCore.py
exists, and will call it.
You can browse through the existing hooks in the
hooks
folder of the PyInstaller distribution folder
and see the names of the packages for which hooks have been written.
Additional hooks are provided by the pyinstaller-hooks-contrib
package, which is typically installed as part of PyInstaller dependencies.
See here
to browse PyInstaller-provided hooks in the online repository,
and here
for hooks provided by the pyinstaller-hooks-contrib
.
Many hooks consist of only one statement, an assignment to hiddenimports
.
For example, the xml.dom
module from Python standard library imports
a module called xml.dom.domreg
, which in turn indirectly imports
xml.dom.minidom
as one of registered XML DOM implementations.
Therefore, to ensure that this implementation module
is collected, PyInstaller provides a hook called
hook-xml.dom.domreg.py
, which contains only the following statement:
hiddenimports = ["xml.dom.minidom"]
When Analysis sees an import xml.dom
statement in the user code (or
one of its dependencies), and subsequently sees that xml.dom
module
imports the xml.dom.domreg
module (via the
from .domreg import getDOMImplementation, registerDOMImplementation
statement), it calls hook-xml.dom.domreg.py
, and examines the value
of hiddenimports
hook global variable set by the hook.
As a result, the xml.dom.minidom
module is collected into the frozen
application, as if the xml.dom.domreg
module (or your source script)
contained a direct import xml.dom.minidom
statement.
A hook can also cause the collection of data files or binaries (shared libraries) from a package, collection of metadata for a package, and it can also prevent collection of packages/modules that are imported only from the hooked module or a package. Examples of these actions are shown below.
When the module that needs a hook is useful only to your project,
you can store the hook file(s) somewhere near your source file.
Then specify their location to the pyinstaller
or pyi-makespec
command with the --additional-hooks-dir
option.
If the hook file(s) are at the same level as the script,
the command could be simply:
pyinstaller --additional-hooks-dir=. myscript.py
If you write a hook for a module used by others, please ask the package developer to include the hook with her/his package or send us the hook file so we can include it in the contributed hooks repository.
How a Hook Is Loaded
A hook is a module named hook-full.import.name.py
in a folder where the Analysis object looks for hooks.
Each time Analysis detects an import, it looks for a hook file with
a matching name.
When one is found, Analysis imports the hook’s code into a Python namespace.
This results in the execution of all top-level statements in the hook source,
for example import statements, assignments to global names, and
function definitions.
The names defined by these statements are visible to Analysis
as attributes of the namespace.
Thus a hook is a normal Python script and can use all normal Python facilities.
For example it could test sys.version
and adjust its
assignment to hiddenimports
based on that.
There are many hooks in the PyInstaller installation,
but a much larger collection can be found in the
community hooks package.
Please browse through them for examples.
Providing PyInstaller Hooks with your Package
As a package developer you can provide hooks for PyInstaller
within your package.
This has the major benefit
that you can easily adopt the hooks
when your package changes.
Thus your package’s users don’t need to wait until PyInstaller
might catch up with these changes.
If both PyInstaller and your package provide hooks for some module,
your package’s hooks take precedence,
but can still be overridden by the command line option
--additional-hooks-dir
.
You can tell PyInstaller about the additional hooks
by defining some simple setuptools entry-points
in your package.
Therefore add entries like these to your setup.cfg
:
[options.entry_points]
pyinstaller40 =
hook-dirs = pyi_hooksample.__pyinstaller:get_hook_dirs
tests = pyi_hooksample.__pyinstaller:get_PyInstaller_tests
This defines two entry-points:
pyinstaller40.hook-dirs
for hook registration:This entry point refers to a function that will be invoked with no parameters. It must return a sequence of strings, each element of which provides an additional absolute path to search for hooks. This is equivalent to passing the
--additional-hooks-dir
command-line option to PyInstaller for each string in the sequence.In this example, the function is
get_hook_dirs() -> List[str]
.pyinstaller40.tests
for test registration:This entry point refers to a function that will be invoked with no parameters. It must return a sequence of strings, each element of which provides an additional absolute path to a directory tree or to a Python source file. These paths are then passed to pytest for test discovery. This allows both testing by this package and by PyInstaller.
In this project, the function is
get_PyInstaller_tests() -> List[str]
.
A sample project providing a guide for integrating PyInstaller hooks and tests into a package is available at https://github.com/pyinstaller/hooksample. This project demonstrates defining a library which includes PyInstaller hooks along with tests for those hooks and sample file for integration into CD/CI testing. Detailed documentation about this sample project is available at https://pyinstaller-sample-hook.readthedocs.io/en/latest/.
Hook Global Variables
A majority of the existing hooks consist entirely of assignments of values to one or more of the following global variables. If any of these are defined by the hook, Analysis takes their values and applies them to the bundle being created.
hiddenimports
A list of module names (relative or absolute) that should be part of the bundled app. This has the same effect as the
--hidden-import
command line option, but it can contain a list of names and is applied automatically only when the hooked module is imported. Example:hiddenimports = ['_gdbm', 'socket', 'h5py.defs']
excludedimports
A list of absolute module names that should not be part of the bundled app. If an excluded module is imported only by the hooked module or one of its sub-modules, the excluded name and its sub-modules will not be part of the bundle. (If an excluded name is explicitly imported in the source file or some other module, it will be kept.) Several hooks use this to prevent automatic inclusion of the
tkinter
module. Example:excludedimports = ['tkinter']
datas
A list of files to bundle with the app as data. Each entry in the list is a tuple containing two strings. The first string specifies a file (or file “glob”) in this system, and the second specifies the name(s) the file(s) are to have in the bundle. (This is the same format as used for the
datas=
argument, see Adding Data Files.) Example:datas = [ ('/usr/share/icons/education_*.png', 'icons') ]
If you need to collect multiple directories or nested directories, you can use helper functions from the
PyInstaller.utils.hooks
module (see below) to create this list, for example:datas = collect_data_files('submodule1') datas += collect_data_files('submodule2')
In rare cases you may need to apply logic to locate particular files within the file system, for example because the files are in different places on different platforms or under different versions. Then you can write a
hook()
function as described below under The hook(hook_api) Function.binaries
A list of files or directories to bundle as binaries. The format is the same as
datas
(tuples with strings that specify the source and the destination). Binaries is a special case ofdatas
, in that PyInstaller will check each file to see if it depends on other dynamic libraries. Example:binaries = [ ('C:\\Windows\\System32\\*.dll', 'dlls') ]
Many hooks use helpers from the
PyInstaller.utils.hooks
module to create this list (see below):binaries = collect_dynamic_libs('zmq')
warn_on_missing_hiddenimports
A boolean flag indicating whether missing hidden imports from the hook (set via
hiddenimports
) should generate warnings or not. By default, missing hidden imports generate warnings, but individual hooks can opt out of this behavior by setting this variable toFalse
. Example:warn_on_missing_hiddenimports = False
module_collection_mode
A setting controlling the collection mode for module(s). The value can be either a string or a dictionary.
When set to a string, the variable controls the collection mode for the hooked package/module. Valid values are:
'pyz'
: collect byte-compiled modules into the embedded PYZ archive. This is the default behavior when no collection mode is specified. If thenoarchive
flag is used withAnalysis
, the PYZ archive is not used, andpyz
collection mode is automatically turned intopyc
one.'pyc'
: collect byte-compiled modules as external data files (as opposed to collecting them into the PYZ archive).'py'
: collect source .py files as external data files. Do not collect byte-compiled modules.'pyz+py'
or'py+pyz'
: collect byte-compiled modules into the embedded PYZ archive and collect corresponding source .py files as external data files.If
noarchive
flag is in effect, the byte-compiled modules are collected as external data files, which causes python to ignore them due to the source files being placed next to them.
The setting is applied to all child modules and subpackages, unless overridden by the setting in their corresponding hook.
Alternatively, the variable can be set to a dictionary comprising module/package names and corresponding collection mode strings. This allows a hook to specify different settings for its main package and subpackages, but also settings for other packages. When multiple hooks provide a setting for the same module name, the end result depends on the hook execution order.
Example:
# hook-mypackage.py # This package must be collected in source form, due to its code # searching for .py files on the filesystem... module_collection_mode = 'py'
Example:
# hook-mypackage.py # Collect only a sub-package / module as source # (without creating a hook for the sub-package). module_collection_mode = { 'mypackage.src_subpackage': 'py' }
Example:
# hook-mypackage.py # Collect whole package as source except for a single sub-package # (without creating a hook for the sub-package). module_collection_mode = { 'mypackage': 'py', 'mypackage.bin_subpackage': 'pyz' }
Example:
# hook-mypackage.py # Force collection of other packages in source form. module_collection_mode = { 'myotherpackage1': 'py', 'myotherpackage2': 'py', }
The ability to control collection mode for other modules/packages from a given hook is intended for cases when the hooked module provides functionality for other modules that requires those other modules to be collected in the source form (for example, JIT compilation available in some deep learning frameworks). However, detection of specific function imports and calls via bytecode scanning requires an access to the modulegraph, and consequently the use of the the hook(hook_api) function. In such cases, the collection mode can be modified using the set_module_collection_mode method from the
hook_api
object instead of setting the global hook variable.
bindepend_symlink_suppression
An option for hooks to prevent the PyInstaller’s binary dependency analysis process from creating a symbolic link to top-level application directory for specific shared library. Has effect only on platforms where such symbolic links are created.
The value can be either a string (single path or pattern), a list of strings, or a set of strings. During binary dependency analysis, the discovered shared library’s source path is matched against all patterns that have been set by hooks to determine whether the symbolic link should be created or not.
This mechanism is intended to be used in specific cases to work around issues caused by symbolic links created by binary dependency analysis; for example, when such a library tries to look up its location, but does not fully resolve the obtained path.
Example:
# hook-mypackage.py import os from PyInstaller import compat from PyInstaller.utils.hooks import get_module_file_attribute # On linux, suppress creation of symbolic links to top-level application # directory for all shared libraries collected from the package's directory. if compat.is_linux: package_dir = os.path.dirname(get_module_file_attribute('mypackage')) bindepend_symlink_suppression = os.path.join(package_dir, "*.so*")
Example:
# hook-mypackage.py from PyInstaller import compat # On linux, suppress creation of symbolic links to top-level application # directory for shared libraries bundled with this package in its two # library subdirectories. if compat.is_linux: bindepend_symlink_suppression = [ "**/mypackage/lib_dir1/*.so*", "**/mypackage/lib_dir2/*.so*", ]
Useful Items in PyInstaller.compat
Various classes and functions to provide some backwards-compatibility with previous versions of Python onward.
A hook may import the following names from PyInstaller.compat
,
for example:
from PyInstaller.compat import base_prefix, is_win
- is_py36, is_py37, is_py38, is_py39, is_py310 is_py311
True when the current version of Python is at least 3.6, 3.7, 3.8, 3.9, or 3.10, 3.11 respectively.
- is_win
True in a Windows system.
- is_cygwin
True when
sys.platform == 'cygwin'
.
- is_darwin
True in macOS.
- is_linux
True in any GNU/Linux system.
- is_solar
True in Solaris.
- is_aix
True in AIX.
- is_freebsd
True in FreeBSD.
- is_openbsd
True in OpenBSD.
- is_venv
True in any virtual environment (either virtualenv or venv).
- base_prefix
String, the correct path to the base Python installation, whether the installation is native or a virtual environment.
- EXTENSION_SUFFIXES
List of Python C-extension file suffixes. Used for finding all binary dependencies in a folder; see
hook-cryptography.py
for an example.
Useful Items in PyInstaller.utils.hooks
A hook may import useful functions from PyInstaller.utils.hooks
.
Use a fully-qualified import statement, for example:
from PyInstaller.utils.hooks import collect_data_files, eval_statement
The functions listed here are generally useful and used in a number of existing hooks.
- exec_statement(statement)
Execute a single Python statement in an externally-spawned interpreter, and return the resulting standard output as a string.
Examples:
tk_version = exec_statement("from _tkinter import TK_VERSION; print(TK_VERSION)") mpl_data_dir = exec_statement("import matplotlib; print(matplotlib.get_data_path())") datas = [ (mpl_data_dir, "") ]
Notes
As of v5.0, usage of this function is discouraged in favour of the new
PyInstaller.isolated
module.
- eval_statement(statement)
Execute a single Python statement in an externally-spawned interpreter, and
eval()
its output (if any).Example:
databases = eval_statement(''' import sqlalchemy.databases print(sqlalchemy.databases.__all__) ''') for db in databases: hiddenimports.append("sqlalchemy.databases." + db)
Notes
As of v5.0, usage of this function is discouraged in favour of the new
PyInstaller.isolated
module.
- check_requirement(requirement)
Check if a PEP 0508 requirement is satisfied. Usually used to check if a package distribution is installed, or if it is installed and satisfies the specified version requirement.
- Parameters:
- Returns:
Boolean indicating whether the requirement is satisfied or not.
- Return type:
Examples
# Assume Pillow 10.0.0 is installed. >>> from PyInstaller.utils.hooks import check_requirement >>> check_requirement('Pillow') True >>> check_requirement('Pillow < 9.0') False >>> check_requirement('Pillow >= 9.0, < 11.0') True
- is_module_satisfies(requirements, version=None, version_attr=None)
A compatibility wrapper for
check_requirement()
, intended for backwards compatibility with existing hooks.In contrast to original implementation from PyInstaller < 6, this implementation only checks the specified PEP 0508 requirement string; i.e., it tries to retrieve the distribution metadata, and compare its version against optional version specifier(s). It does not attempt to fall back to checking the module’s version attribute, nor does it support
version
andversion_attr
arguments.- Parameters:
requirements (str) – Requirements string passed to the
check_requirement()
.version (None) – Deprecated and unsupported. Must be
None
.version_attr (None) – Deprecated and unsupported. Must be
None
.
- Returns:
Boolean indicating whether the requirement is satisfied or not.
- Return type:
- Raises:
ValueError – If either
version
orversion_attr
are specified and are not None.
- collect_all(package_name, include_py_files=True, filter_submodules=<function <lambda>>, exclude_datas=None, include_datas=None, on_error='warn once')
Collect everything for a given package name.
- Parameters:
package_name (
str
) – Animport
-able package name.include_py_files (
bool
) – Forwarded tocollect_data_files()
.filter_submodules (
Callable
) – Forwarded tocollect_submodules()
.exclude_datas (
list
|None
) – Forwarded tocollect_data_files()
.include_datas (
list
|None
) – Forwarded tocollect_data_files()
.on_error (
str
) – Forwarded ontocollect_submodules()
.
- Returns:
A
(datas, binaries, hiddenimports)
triplet containing:All data files, raw Python files (if include_py_files), and distribution metadata directories (if applicable).
All dynamic libraries as returned by
collect_dynamic_libs()
.All submodules of package_name.
- Return type:
Typical use:
datas, binaries, hiddenimports = collect_all('my_package_name')
- collect_submodules(package, filter=<function <lambda>>, on_error='warn once')
List all submodules of a given package.
- Parameters:
package (
str
) – Animport
-able package.filter (
Callable
[[str
],bool
]) – Filter the submodules found: A callable that takes a submodule name and returns True if it should be included.on_error (
str
) –The action to take when a submodule fails to import. May be any of:
raise: Errors are reraised and terminate the build.
warn: Errors are downgraded to warnings.
warn once: The first error issues a warning but all subsequent errors are ignored to minimise stderr pollution. This is the default.
ignore: Skip all errors. Don’t warn about anything.
- Returns:
All submodules to be assigned to
hiddenimports
in a hook.
This function is intended to be used by hook scripts, not by main PyInstaller code.
Examples:
# Collect all submodules of Sphinx don't contain the word ``test``. hiddenimports = collect_submodules( "Sphinx", ``filter=lambda name: 'test' not in name)
Changed in version 4.5: Add the on_error parameter.
- is_module_or_submodule(name, mod_or_submod)
This helper function is designed for use in the
filter
argument ofcollect_submodules()
, by returningTrue
if the givenname
is a module or a submodule ofmod_or_submod
.Examples
The following excludes
foo.test
andfoo.test.one
but notfoo.testifier
.collect_submodules('foo', lambda name: not is_module_or_submodule(name, 'foo.test'))``
- is_package(module_name)
Check if a Python module is really a module or is a package containing other modules, without importing anything in the main process.
- Parameters:
module_name (
str
) – Module name to check.- Returns:
True if module is a package else otherwise.
- collect_data_files(package, include_py_files=False, subdir=None, excludes=None, includes=None)
This function produces a list of
(source, dest)
entries for data files that reside inpackage
. Its output can be directly assigned todatas
in a hook script; for example, seehook-sphinx.py
. The data files are all files that are not shared libraries / binary python extensions (based on extension check) and are not python source (.py) files or byte-compiled modules (.pyc). Collection of the .py and .pyc files can be toggled via theinclude_py_files
flag. Parameters:The
package
parameter is a string which names the package.By default, python source files and byte-compiled modules (files with
.py
and.pyc
suffix) are not collected; setting theinclude_py_files
argument toTrue
collects these files as well. This is typically used when a package requires source .py files to be available; for example, JIT compilation used in deep-learning frameworks, code that requires access to .py files (for example, to check their date), or code that tries to extend sys.path with subpackage paths in a way that is incompatible with PyInstaller’s frozen importer.. However, in contemporary PyInstaller versions, the preferred way of collecting source .py files is by using the module collection mode setting (which enables collection of source .py files in addition to or in lieu of collecting byte-compiled modules into PYZ archive).The
subdir
argument gives a subdirectory relative topackage
to search, which is helpful when submodules are imported at run-time from a directory lacking__init__.py
.The
excludes
argument contains a sequence of strings or Paths. These provide a list of globs to exclude from the collected data files; if a directory matches the provided glob, all files it contains will be excluded as well. All elements must be relative paths, which are relative to the provided package’s path (/subdir
if provided).Therefore,
*.txt
will exclude only.txt
files inpackage
‘s path, while**/*.txt
will exclude all.txt
files inpackage
‘s path and all its subdirectories. Likewise,**/__pycache__
will exclude all files contained in any subdirectory named__pycache__
.The
includes
function likeexcludes
, but only include matching paths.excludes
overrideincludes
: a file or directory in both lists will be excluded.
This function does not work on zipped Python eggs.
This function is intended to be used by hook scripts, not by main PyInstaller code.
- collect_dynamic_libs(package, destdir=None, search_patterns=['*.dll', '*.dylib', 'lib*.so'])
This function produces a list of (source, dest) of dynamic library files that reside in package. Its output can be directly assigned to
binaries
in a hook script. The package parameter must be a string which names the package.
- get_module_file_attribute(package)
Get the absolute path to the specified module or package.
Modules and packages must not be directly imported in the main process during the analysis. Therefore, to avoid leaking the imports, this function uses an isolated subprocess when it needs to import the module and obtain its
__file__
attribute.
- get_module_attribute(module_name, attr_name)
Get the string value of the passed attribute from the passed module if this attribute is defined by this module _or_ raise AttributeError otherwise.
Since modules cannot be directly imported during analysis, this function spawns a subprocess importing this module and returning the string value of this attribute in this module.
- Parameters:
- Returns:
String value of this attribute.
- Return type:
- Raises:
AttributeError – If this attribute is undefined.
- get_package_paths(package)
Given a package, return the path to packages stored on this machine and also returns the path to this particular package. For example, if pkg.subpkg lives in /abs/path/to/python/libs, then this function returns
(/abs/path/to/python/libs, /abs/path/to/python/libs/pkg/subpkg)
.NOTE: due to backwards compatibility, this function returns only one package path along with its base directory. In case of PEP 420 namespace package with multiple location, only first location is returned. To obtain all package paths, use the
get_all_package_paths
function and obtain corresponding base directories using thepackage_base_path
helper.
- copy_metadata(package_name, recursive=False)
Collect distribution metadata so that
importlib.metadata.distribution()
orpkg_resources.get_distribution()
can find it.This function returns a list to be assigned to the
datas
global variable. This list instructs PyInstaller to copy the metadata for the given package to the frozen application’s data directory.- Parameters:
- Returns:
This should be assigned to
datas
.- Return type:
Examples
>>> from PyInstaller.utils.hooks import copy_metadata >>> copy_metadata('sphinx') [('c:\python27\lib\site-packages\Sphinx-1.3.2.dist-info', 'Sphinx-1.3.2.dist-info')]
Some packages rely on metadata files accessed through the
importlib.metadata
(or the now-deprecatedpkg_resources
) module. PyInstaller does not collect these metadata files by default. If a package fails without the metadata (either its own, or of another package that it depends on), you can use this function in a hook to collect the corresponding metadata files into the frozen application. The tuples in the returned list contain two strings. The first is the full path to the package’s metadata directory on the system. The second is the destination name, which typically corresponds to the basename of the metadata directory. Adding these tuples the thedatas
hook global variable, the metadata is collected into top-level application directory (where it is usually searched for).Changed in version 4.3.1: Prevent
dist-info
metadata folders being renamed toegg-info
which brokepkg_resources.require
with extras (see #3033).Changed in version 4.4.0: Add the recursive option.
- collect_entry_point(name)
Collect modules and metadata for all exporters of a given entry point.
- Parameters:
name (
str
) – The name of the entry point. Check the documentation for the library that uses the entry point to find its name.- Returns:
A
(datas, hiddenimports)
pair that should be assigned to thedatas
andhiddenimports
, respectively.
For libraries, such as
pytest
orkeyring
, that rely on plugins to extend their behaviour.Examples
Pytest uses an entry point called
'pytest11'
for its extensions. To collect all those extensions use:datas, hiddenimports = collect_entry_point("pytest11")
These values may be used in a hook or added to the
datas
andhiddenimports
arguments in the.spec
file. See Using Spec Files.Added in version 4.3.
- get_homebrew_path(formula='')
Return the homebrew path to the requested formula, or the global prefix when called with no argument.
Returns the path as a string or None if not found.
- include_or_exclude_file(filename, include_list=None, exclude_list=None)
Generic inclusion/exclusion decision function based on filename and list of include and exclude patterns.
- Parameters:
- Returns:
A boolean indicating whether the file should be included or not.
If
include_list
is provided, True is returned only if the filename matches one of include patterns (and does not match any patterns inexclude_list
, if provided). Ifinclude_list
is not provided, True is returned if filename does not match any patterns inexclude list
, if provided. If neither list is provided, True is returned for any filename.
- collect_delvewheel_libs_directory(package_name, libdir_name=None, datas=None, binaries=None)
Collect data files and binaries from the .libs directory of a delvewheel-enabled python wheel. Such wheels ship their shared libraries in a .libs directory that is located next to the package directory, and therefore falls outside the purview of the collect_dynamic_libs() utility function.
- Parameters:
package_name – Name of the package (e.g., scipy).
libdir_name – Optional name of the .libs directory (e.g., scipy.libs). If not provided, “.libs” is added to
package_name
.datas – Optional list of datas to which collected data file entries are added. The combined result is retuned as part of the output tuple.
binaries – Optional list of binaries to which collected binaries entries are added. The combined result is retuned as part of the output tuple.
- Returns:
A
(datas, binaries)
pair that should be assigned to thedatas
andbinaries
, respectively.- Return type:
Examples
Collect the
scipy.libs
delvewheel directory belonging to the Windowsscipy
wheel:datas, binaries = collect_delvewheel_libs_directory("scipy")
When the collected entries should be added to existing
datas
andbinaries
listst, the following form can be used to avoid using intermediate temporary variables and merging those into existing lists:datas, binaries = collect_delvewheel_libs_directory("scipy", datas=datas, binaries=binaries)
Added in version 5.6.
Support for Conda
Additional helper methods for working specifically with Anaconda distributions are found at
PyInstaller.utils.hooks.conda_support
which is designed to mimic (albeit loosely) the importlib.metadata package. These functions find and parse the
distribution metadata from json files located in the conda-meta
directory.
Added in version 4.2.0.
This module is available only if run inside a Conda environment. Usage of this module should therefore be wrapped in a conditional clause:
from PyInstaller.compat import is_pure_conda
if is_pure_conda:
from PyInstaller.utils.hooks import conda_support
# Code goes here. e.g.
binaries = conda_support.collect_dynamic_libs("numpy")
...
Packages are all referenced by the distribution name you use to install it, rather than the package name you import
it with. I.e., use distribution("pillow")
instead of distribution("PIL")
or use package_distribution("PIL")
.
- distribution(name)
Get distribution information for a given distribution name (i.e., something you would
conda install
).- Return type:
- package_distribution(name)
Get distribution information for a package (i.e., something you would import).
- Return type:
For example, the package
pkg_resources
belongs to the distributionsetuptools
, which contains three packages.>>> package_distribution("pkg_resources") Distribution(name="setuptools", packages=['easy_install', 'pkg_resources', 'setuptools'])
- files(name, dependencies=False, excludes=None)
List all files belonging to a distribution.
- Parameters:
- Return type:
- Returns:
All filenames belonging to the given distribution.
With
dependencies=False
, this is just a shortcut for:conda_support.distribution(name).files
- requires(name, strip_versions=False)
List requirements of a distribution.
- class Distribution(json_path)
A bucket class representation of a Conda distribution.
This bucket exports the following attributes:
- Variables:
name – The distribution’s name.
version – Its version.
files – All filenames as
PackagePath()
s included with this distribution.dependencies – Names of other distributions that this distribution depends on (with version constraints removed).
packages – Names of importable packages included in this distribution.
This class is not intended to be constructed directly by users. Rather use
distribution()
orpackage_distribution()
to provide one for you.
- class PackagePath(*args, **kwargs)
A filename relative to Conda’s root (
sys.prefix
).This class inherits from
pathlib.PurePosixPath
even on non-Posix OSs. To convert to apathlib.Path
pointing to the real file, use thelocate()
method.- locate()
Return a path-like object for this path pointing to the file’s true location.
- walk_dependency_tree(initial, excludes=None)
Collect a
Distribution
and all direct and indirect dependencies of that distribution.- Parameters:
- Returns:
A
{name: distribution}
mapping wheredistribution
is the output ofconda_support.distribution(name)
.
- collect_dynamic_libs(name, dest='.', dependencies=True, excludes=None)
Collect DLLs for distribution name.
- Parameters:
- Returns:
List of DLLs in PyInstaller’s
(source, dest)
format.
This collects libraries only from Conda’s shared
lib
(Unix) orLibrary/bin
(Windows) folders. To collect from inside a distribution’s installation use the regularPyInstaller.utils.hooks.collect_dynamic_libs()
.
Subprocess isolation with PyInstaller.isolated
PyInstaller hooks typically will need to import the package which they are written for but doing so may manipulate
globals such as sys.path
or os.environ
in ways that affect the build. For example, on Windows,
Qt’s binaries are added to then loaded via PATH
in such a way that if you import multiple Qt variants in one
session then there is no guarantee which variant’s binaries each variant will get!
To get around this, PyInstaller does any such tasks in an isolated Python subprocess and ships a
PyInstaller.isolated
submodule to do so in hooks.
from PyInstaller import isolated
This submodule provides:
isolated.call()
to evaluate functions in isolation.@isolated.decorate
to mark a function as always called in isolation.isolated.Python()
to efficiently call many functions in a single child instance of Python.
- call(function, *args, **kwargs)
Call a function with arguments in a separate child Python. Retrieve its return value.
- Parameters:
function – The function to send and invoke.
*args
**kwargs – Positional and keyword arguments to send to the function. These must be simple builtin types - not custom classes.
- Returns:
The return value of the function. Again, these must be basic types serialisable by
marshal.dumps()
.- Raises:
RuntimeError – Any exception which happens inside an isolated process is caught and reraised in the parent process.
To use, define a function which returns the information you’re looking for. Any imports it requires must happen in the body of the function. For example, to safely check the output of
matplotlib.get_data_path()
use:# Define a function to be ran in isolation. def get_matplotlib_data_path(): import matplotlib return matplotlib.get_data_path() # Call it with isolated.call(). get_matplotlib_data_path = isolated.call(matplotlib_data_path)
For single use functions taking no arguments like the above you can abuse the decorator syntax slightly to define and execute a function in one go.
>>> @isolated.call ... def matplotlib_data_dir(): ... import matplotlib ... return matplotlib.get_data_path() >>> matplotlib_data_dir '/home/brenainn/.pyenv/versions/3.9.6/lib/python3.9/site-packages/matplotlib/mpl-data'
Functions may take positional and keyword arguments and return most generic Python data types.
>>> def echo_parameters(*args, **kwargs): ... return args, kwargs >>> isolated.call(echo_parameters, 1, 2, 3) (1, 2, 3), {} >>> isolated.call(echo_parameters, foo=["bar"]) (), {'foo': ['bar']}
Notes
To make a function behave differently if it’s isolated, check for the
__isolated__
global.if globals().get("__isolated__", False): # We're inside a child process. ... else: # This is the master process. ...
- decorate(function)
Decorate a function so that it is always called in an isolated subprocess.
Examples
To use, write a function then prepend
@isolated.decorate
.@isolated.decorate def add_1(x): '''Add 1 to ``x``, displaying the current process ID.''' import os print(f"Process {os.getpid()}: Adding 1 to {x}.") return x + 1
The resultant
add_1()
function can now be called as you would a normal function and it’ll automatically use a subprocess.>>> add_1(4) Process 4920: Adding 1 to 4. 5 >>> add_1(13.2) Process 4928: Adding 1 to 13.2. 14.2
- class Python(strict_mode=None)
Start and connect to a separate Python subprocess.
This is the lowest level of public API provided by this module. The advantage of using this class directly is that it allows multiple functions to be evaluated in a single subprocess, making it faster than multiple calls to
call()
.The
strict_mode
argument controls behavior when the child process fails to shut down; if strict mode is enabled, an error is raised, otherwise only warning is logged. If the value ofstrict_mode
isNone
, the value ofPyInstaller.compat.strict_collect_mode
is used (which in turn is controlled by thePYINSTALLER_STRICT_COLLECT_MODE
environment variable.Examples
To call some predefined functions
x = foo()
,y = bar("numpy")
andz = bazz(some_flag=True)
all using the same isolated subprocess use:with isolated.Python() as child: x = child.call(foo) y = child.call(bar, "numpy") z = child.call(bazz, some_flag=True)
The hook(hook_api)
Function
In addition to, or instead of, setting global values,
a hook may define a function hook(hook_api)
.
A hook()
function should only be needed if the hook
needs to apply sophisticated logic or to make a complex
search of the source machine.
The Analysis object calls the function and passes it a hook_api
object
which has the following immutable properties:
__name__
:The fully-qualified name of the module that caused the hook to be called, e.g.,
six.moves.tkinter
.__file__
:The absolute path of the module. If it is:
A standard (rather than namespace) package, this is the absolute path of this package’s directory.
A namespace (rather than standard) package, this is the abstract placeholder
-
.A non-package module or C extension, this is the absolute path of the corresponding file.
__path__
:A list of the absolute paths of all directories comprising the module if it is a package, or
None
. Typically the list contains only the absolute path of the package’s directory.co
:Code object compiled from the contents of
__file__
(e.g., via thecompile()
builtin).analysis
:The
Analysis
object that loads the hook.
The hook_api
object also offers the following methods:
add_imports( *names )
:The
names
argument may be a single string or a list of strings giving the fully-qualified name(s) of modules to be imported. This has the same effect as adding the names to thehiddenimports
global.add_datas( tuple_list )
:The
tuple_list
argument has the format used with thedatas
global variable. This call has the effect of adding items to that list.add_binaries( tuple_list )
:The
tuple_list
argument has the format used with thebinaries
global variable. This call has the effect of adding items to that list.
set_module_collection_mode ( name, mode )
:Set the package collection mode for the specified package/module name. Valid values for
mode
are:'pyz'
,'pyc'
,'py'
,'pyz+py'
,'py+pyz'
andNone
.None
clears/resets the setting for the given package/module name - but only within the current hook’s context! The collection mode may be set for the hooked package, its sub-module or sub-package, or for other packages. Ifname
isNone
, it is substituted with the hooked package/module name.add_bindepend_symlink_suppression_pattern( pattern )
:Add the given path or path pattern to the set of patterns that prevent binary dependency analysis from creating a symbolic link to the top-level application directory. The same can be achieved by setting the bindepend_symlink_suppression hook global variable.
The hook()
function can add, remove or change included files using the
above methods of hook_api
.
Or, it can simply set values in the four global variables, because
these will be examined after hook()
returns.
Hooks may access the user parameters, given in the hooksconfig
argument in
the spec file, by calling get_hook_config()
inside a hook() function.
- get_hook_config(hook_api, module_name, key)
Get user settings for hooks.
- Parameters:
- Returns:
The value for the config.
None
if not set.
The
get_hook_config
function will lookup settings in theAnalysis.hooksconfig
dict.The hook settings can be added to
.spec
file in the form of:a = Analysis(["my-app.py"], ... hooksconfig = { "gi": { "icons": ["Adwaita"], "themes": ["Adwaita"], "languages": ["en_GB", "zh_CN"], }, }, ... )
The pre_find_module_path( pfmp_api )
Method
You may write a hook with the special function pre_find_module_path( pfmp_api )
.
This method is called when the hooked module name is first seen
by Analysis, before it has located the path to that module or package
(hence the name “pre-find-module-path”).
Hooks of this type are only recognized if they are stored in
a sub-folder named pre_find_module_path
in a hooks folder,
either in the distributed hooks folder or an --additional-hooks-dir
folder.
You may have normal hooks as well as hooks of this type for the same module.
For example PyInstaller includes both a hooks/hook-distutils.py
and also a hooks/pre_find_module_path/hook-distutils.py
.
The pfmp_api
object that is passed has the following immutable attribute:
module_name
:A string, the fully-qualified name of the hooked module.
The pfmp_api
object has one mutable attribute, search_dirs
.
This is a list of strings that specify the absolute path, or paths,
that will be searched for the hooked module.
The paths in the list will be searched in sequence.
The pre_find_module_path()
function may replace or change
the contents of pfmp_api.search_dirs
.
Immediately after return from pre_find_module_path()
, the contents
of search_dirs
will be used to find and analyze the module.
For an example of use,
see the file hooks/pre_find_module_path/hook-distutils.py
.
It uses this method to redirect a search for distutils when
PyInstaller is executing in a virtual environment.
The pre_safe_import_module( psim_api )
Method
You may write a hook with the special function pre_safe_import_module( psim_api )
.
This method is called after the hooked module has been found,
but before it and everything it recursively imports is added
to the “graph” of imported modules.
Use a pre-safe-import hook in the unusual case where:
The script imports package.dynamic-name
The package exists
however, no module dynamic-name exists at compile time (it will be defined somehow at run time)
You use this type of hook to make dynamically-generated names known to PyInstaller. PyInstaller will not try to locate the dynamic names, fail, and report them as missing. However, if there are normal hooks for these names, they will be called.
Hooks of this type are only recognized if they are stored in a sub-folder
named pre_safe_import_module
in a hooks folder,
either in the distributed hooks folder or an --additional-hooks-dir
folder.
(See the distributed hooks/pre_safe_import_module
folder for examples.)
You may have normal hooks as well as hooks of this type for the same module.
For example the distributed system has both a hooks/hook-gi.repository.GLib.py
and also a hooks/pre_safe_import_module/hook-gi.repository.GLib.py
.
The psim_api
object offers the following attributes,
all of which are immutable (an attempt to change one raises an exception):
module_basename
:String, the unqualified name of the hooked module, for example
text
.module_name
:String, the fully-qualified name of the hooked module, for example
email.mime.text
.module_graph
:The module graph representing all imports processed so far.
parent_package
:If this module is a top-level module of its package,
None
. Otherwise, the graph node that represents the import of the top-level module.
The last two items, module_graph
and parent_package
,
are related to the module-graph, the internal data structure used by
PyInstaller to document all imports.
Normally you do not need to know about the module-graph.
The psim_api
object also offers the following methods:
add_runtime_module( fully_qualified_name )
:Use this method to add an imported module whose name may not appear in the source because it is dynamically defined at run-time. This is useful to make the module known to PyInstaller and avoid misleading warnings. A typical use applies the name from the
psim_api
:psim_api.add_runtime_module( psim_api.module_name )
add_alias_module( real_module_name, alias_module_name )
:real_module_name
is the fully-qualifed name of an existing module, one that has been or could be imported by name (it will be added to the graph if it has not already been imported).alias_module_name
is a name that might be referenced in the source file but should be treated as if it werereal_module_name
. This method ensures that if PyInstaller processes an import ofalias_module_name
it will usereal_module_name
.append_package_path( directory )
:The hook can use this method to add a package path to be searched by PyInstaller, typically an import path that the imported module would add dynamically to the path if the module was executed normally.
directory
is a string, a pathname to add to the__path__
attribute.