Troubleshooting

Common problems encountered when running HEDGEHOG, with explanations and solutions.

missing required 'smiles' column

Symptom: input loading fails with an error that the file is missing the required smiles column.

Cause: the input was parsed as CSV/TSV but does not contain a smiles header, or the file is not in the expected molecule-table format.

Fix: use CSV/TSV with a smiles column:

smiles,model_name
CCO,demo
CCN,demo

For headerless .smi files, keep one SMILES per line and use an optional second token for model_name.

GNINA/SMINA Binary Not Found

Symptom: docking fails before producing gnina_out.sdf or smina_out.sdf.

Cause: docking is enabled but the selected binary cannot be resolved from PATH or from the explicit bin value in config_docking.yml.

Fix:

1. Install the binary and make it available on PATH, or set an explicit path:

gnina_config:
  bin: /path/to/gnina

No Docking Output Found

Symptom: logs report that docking finished but no results were detected.

Check first:

1. selected docking tool in config_docking.yml
2. receptor and autobox/reference ligand paths
3. generated scripts under stages/05_docking/_workdir/
4. binary path and executable permissions
5. auto_run setting

The expected final artifacts are tool-specific SDF files such asstages/05_docking/smina/smina_out.sdf, stages/05_docking/gnina/gnina_out.sdf, or stages/05_docking/matcha/matcha_out.sdf.

Docking Script Path Issues

Docking scripts generated by HEDGEHOG execute from a _workdir/ subdirectory inside the docking stage output, not from the stage directory itself. This causes path resolution issues if relative paths are used.

Three specific path bugs to watch for:

1. Config reference — the docking config file lives in the pipeline’s root working directory. Use an absolute path or config_file.relative_to(ligands_dir) instead of just the file name.
2. External prep output path argument — the output SDF path must be absolute, otherwise it resolves relative to _workdir/, creating nested _workdir/_workdir/ paths.
3. External prep input path argument — same issue as above; the input CSV path must be absolute.

Symptoms: docking jobs fail with FileNotFoundError, or output files appear in unexpected nested directories.

Fix: ensure all paths passed to docking scripts are absolute:

# Wrong: relative path breaks when cwd is _workdir/
script_args = [str(output_sdf)]
 
# Correct: always resolve to absolute
script_args = [str(output_sdf.resolve())]

Conda / Mamba Detection (GNINA)

If you configure GNINA to run inside an existing conda environment, HEDGEHOG may need to locate conda.sh to activate that environment before launching GNINA. When conda_sh is not provided in your docking configuration, HEDGEHOG auto-detects common conda installs by searching for these directory names under the user’s home:

• miniforge
• miniconda3
• mambaforge
• anaconda3

Symptom: docking (GNINA) stage fails with an error like conda not found or CondaError, or GNINA cannot find required shared libraries after activation.

Fix: ensure one of the supported conda distributions is installed and its bin/ directory is on your PATH, or set conda_sh explicitly to your .../etc/profile.d/conda.sh path.

# Verify conda is accessible
conda --version
 
# If using miniforge, conda.sh is typically here:
ls ~/miniforge/etc/profile.d/conda.sh

Autobox Coordinate Frame Mismatch

When using autobox docking (where the search box is derived from a reference ligand SDF), the reference ligand must be in the same coordinate frame as the receptor PDB file.

Symptom: docking scores are unreasonably poor, or all poses are placed far from the binding site.

Common cause: the reference ligand was extracted from an apo (ligand-free) crystal structure, but the receptor PDB comes from a holo (ligand-bound) structure with a different coordinate frame. Even small translations or rotations between crystal structures cause the autobox to miss the binding pocket entirely.

Fix: superimpose the apo and holo structures before extracting the reference ligand, or use a reference ligand from the same PDB file as the receptor:

from Bio.PDB import PDBParser, Superimposer
 
parser = PDBParser(QUIET=True)
ref_struct = parser.get_structure("holo", "receptor_holo.pdb")
mobile_struct = parser.get_structure("apo", "receptor_apo.pdb")
 
# Superimpose using CA atoms, then transform the reference ligand
sup = Superimposer()
sup.set_atoms(ref_atoms, mobile_atoms)
sup.apply(mobile_struct.get_atoms())

External Tool Licensing

The optional ligand preparation and protein preparation steps may use proprietary third-party tools. These often require a valid vendor license.

Symptom: pipeline fails at input preprocessing with license checkout failures or vendor-tool initialization errors.

Fix:

1. Set the environment variable(s) required by your tool vendor to point to the installation directory:

export TOOL_HOME=/opt/proprietary_tools/suite2024-1

2. Verify the license server is accessible:

$TOOL_HOME/licadmin STAT

3. If you do not have a valid license, remove or clear the ligand_preparation_tool and protein_preparation_tool fields in config.yml. HEDGEHOG will skip external preprocessing and rely on the Mol Prep stage (Datamol standardization), which does not require a license:

# config.yml — disable external preparation tools
ligand_preparation_tool:
protein_preparation_tool:

Memory Issues with Large SDF Files

Docking stages load and process SDF files that can grow very large when docking thousands of molecules. This may cause out-of-memory errors, especially on systems with limited RAM.

Symptoms: the process is killed by the OS (Killed or OOM), or you see MemoryError in the log.

Mitigations:

1. Reduce input size — use the sample_size parameter in config.yml to cap the number of molecules entering the pipeline:

sample_size: 500  # Process at most 500 molecules

2. Split input files — divide your input CSV into smaller batches and run the pipeline on each batch separately.

3. Reduce parallelism — lower n_jobs to reduce peak memory from concurrent docking processes:

n_jobs: 8  # Default is often set to all cores

4. Monitor memory — watch system memory during the docking stage:

# In a separate terminal
watch -n 5 free -h

AiZynthFinder Installation

The retrosynthesis stage requires AiZynthFinder. Install it with the built-in setup command.

Symptom: synthesis stage fails with ModuleNotFoundError: aizynthfinder or the retrosynthesis subprocess exits immediately.

Fix:

1. Run setup from the project root:

uv run hedgehog setup aizynthfinder

This command installs the optional retrosynthesis extra into the project environment and downloads the required public data (model files and templates) into modules/aizynthfinder/.

2. Verify the installation:

# Check that public data was downloaded
ls modules/aizynthfinder/public/
 
# Check that logging config exists
ls modules/aizynthfinder/aizynthfinder/data/

3. If setup fails during dependency sync, verify your local uv installation, Python version compatibility (AiZynthFinder currently supports Python 3.10-3.12), and outbound package-index access.

AiZynthFinder Worker Failures

Symptom: AiZynthFinder reports Not all output files produced, or a long synthesis run ends with no combined JSON file.

Hedgehog-managed isolated shards are enabled by default. Successful molecule checkpoints are retained and only failed shards are retried. Inspect aizynthfinder_run_manifest.json for failed shard return codes and log paths.

An ONNX Runtime warning about failing to inspect /sys/class/drm/card0 only means that GPU discovery is unavailable. Hedgehog’s AiZynthFinder worker suppresses this warning and continues CPU inference; ONNX errors are still logged.

If the host already has stopped AiZ processes, inspect them before starting another memory-heavy run:

ps -eo pid,stat,rss,args | grep aizynth
free -h

A process state beginning with T is suspended, often after Ctrl-Z, and can retain model and search-tree memory. Resume or terminate only the jobs you own. Lower synthesis n_jobs when swap usage or per-worker memory is high.

Shepherd-Score on Python 3.13

Symptom: installing Shepherd dependencies fails on Python 3.13 with wheel or open3d errors.

open3d (required by shepherd-score) does not publish wheels for some Python ABIs (notably cp313), so installing Shepherd dependencies in the main environment can fail.

HEDGEHOG base install is decoupled from Shepherd dependencies. Use an isolated worker environment:

uv run hedgehog setup shepherd-worker --yes

If Shepherd backend is unavailable at runtime, docking filters soft-skip Shepherd and log a warning with setup instructions.

TUI Port Conflicts

The TUI (Text User Interface) uses a JSON-RPC protocol over stdio to communicate between the Node.js frontend and the Python backend. The backend is launched as a child process — it does not bind to a network port by default.

However, the Node.js development server (used during TUI development) binds to a local port. If that port is already in use, the TUI will fail to start.

Symptom: Error: listen EADDRINUSE when launching the TUI in development mode.

Fix:

1. Find and kill the process using the port:

# Find what is using port 3000
lsof -i :3000
 
# Kill the process
kill <PID>

2. For production use, launch the TUI via the CLI command, which uses the built bundle and stdio communication without binding any port:

uv run hedgehog tui

3. If the TUI has not been built yet, the CLI will automatically run npm install && npm run build inside the tui/ directory before launching.

TUI Cancellation Is Not Immediate

Symptom: pressing c in the TUI requests cancellation, but an external docking or synthesis command continues for a while.

Cause: cancellation is cooperative. The pipeline stops at safe checkpoints, and long-running external subprocesses may not terminate immediately.

Fix: wait for the current stage checkpoint and inspect the run log. If a separate external process must be stopped manually, identify it from the stage work directory or system process list before killing it.

Environment-Specific Docking Config

The default docking configuration file (config_docking.yml) ships with repository-relative example receptor and reference-ligand paths. Production runs should replace them with target-specific files.

Symptom: docking stage fails immediately with FileNotFoundError for receptor PDB or reference ligand files.

Fix: copy the bundled configs into your project directory, pass the copied master config with --config, and update receptor/reference ligand paths:

receptor_pdb: /path/to/your/receptor.pdb
gnina_config:
  autobox_ligand: /path/to/your/reference_ligand.sdf

Common Log Messages