I’m loving the Framework! However, I’m currently struggling with a Python data loader that works perfectly on an Intel Mac but fails on an M3 Mac.
Reproduction steps:
I downloaded the “loader-python-to-csv” example from https://github.com/observablehq/framework/tree/main/examples/loader-python-to-csv and followed these steps:
npm ipython3 -m venv .venvsource .venv/bin/activatepip install -r requirements.txtnpm run dev
(I performed the same steps on two Macs: one Intel, the other M3.)
On the Intel Mac, everything runs correctly:
However, on the M3 Mac, I encounter the following error:
Here are the errors in the terminal:
GET /_file/data/predictions.csv?sha=ea2ab67ab4b0572ca91d786ba2e5823090ca9081acdcab99cbc3694af7f6c1ce
load src/data/predictions.csv.py → [missing] Traceback (most recent call last):
File "/Users/user/Documents/Observable/framework/examples/loader-python-to-csv/src/data/predictions.csv.py", line 1, in <module>
import pandas as pd
File "/Users/user/Documents/Observable/framework/examples/loader-python-to-csv/.venv/lib/python3.10/site-packages/pandas/__init__.py", line 19, in <module>
raise ImportError(
ImportError: Unable to import required dependencies:
numpy: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.
error in 96ms: loader exited with code 1
Error: loader exited with code 1
at CommandLoader.exec (file:///Users/user/Documents/Observable/framework/examples/loader-python-to-csv/node_modules/@observablehq/framework/dist/dataloader.js:297:13)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async file:///Users/user/Documents/Observable/framework/examples/loader-python-to-csv/node_modules/@observablehq/framework/dist/dataloader.js:240:11
I suspect there’s a conflict with the Python packages (specifically, numpy) and the Mac architecture. However, when I run the Python script directly from the terminal (within the virtual environment), it works:
python3 src/data/predictions.csv.py
But it fails when the data loader is run from the Observable preview (or build).
Here are the Python packages installed with pip install -r requirements.txt on both architectures:
Intel Mac:
% pip install -r requirements.txt
Collecting joblib==1.3.2
Downloading joblib-1.3.2-py3-none-any.whl (302 kB)
|████████████████████████████████| 302 kB 2.6 MB/s
Collecting numpy==1.26.3
Downloading numpy-1.26.3-cp310-cp310-macosx_10_9_x86_64.whl (20.6 MB)
|████████████████████████████████| 20.6 MB 5.7 MB/s
Collecting pandas==2.2.0
Downloading pandas-2.2.0-cp310-cp310-macosx_10_9_x86_64.whl (12.5 MB)
|████████████████████████████████| 12.5 MB 8.5 MB/s
Collecting python-dateutil==2.8.2
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytz==2023.3.post1
Using cached pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Collecting scikit-learn==1.5.0
Downloading scikit_learn-1.5.0-cp310-cp310-macosx_10_9_x86_64.whl (12.1 MB)
|████████████████████████████████| 12.1 MB 8.1 MB/s
Collecting scipy==1.12.0
Downloading scipy-1.12.0-cp310-cp310-macosx_10_9_x86_64.whl (38.9 MB)
|████████████████████████████████| 38.9 MB 12.4 MB/s
Collecting six==1.16.0
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting threadpoolctl==3.2.0
Downloading threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Collecting tzdata==2023.4
Downloading tzdata-2023.4-py2.py3-none-any.whl (346 kB)
|████████████████████████████████| 346 kB 1.6 MB/s
Installing collected packages: six, numpy, tzdata, threadpoolctl, scipy, pytz, python-dateutil, joblib, scikit-learn, pandas
Successfully installed joblib-1.3.2 numpy-1.26.3 pandas-2.2.0 python-dateutil-2.8.2 pytz-2023.3.post1 scikit-learn-1.5.0 scipy-1.12.0 six-1.16.0 threadpoolctl-3.2.0 tzdata-2023.4
M3 Mac:
% pip install --no-cache-dir -r requirements.txt
Collecting joblib==1.3.2
Downloading joblib-1.3.2-py3-none-any.whl (302 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 302.2/302.2 kB 1.3 MB/s eta 0:00:00
Collecting numpy==1.26.3
Downloading numpy-1.26.3-cp310-cp310-macosx_11_0_arm64.whl (14.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 1.3 MB/s eta 0:00:00
Collecting pandas==2.2.0
Downloading pandas-2.2.0-cp310-cp310-macosx_11_0_arm64.whl (11.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.8/11.8 MB 1.4 MB/s eta 0:00:00
Collecting python-dateutil==2.8.2
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 743.4 kB/s eta 0:00:00
Collecting pytz==2023.3.post1
Downloading pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 502.5/502.5 kB 953.6 kB/s eta 0:00:00
Collecting scikit-learn==1.5.0
Downloading scikit_learn-1.5.0-cp310-cp310-macosx_12_0_arm64.whl (11.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.0/11.0 MB 1.7 MB/s eta 0:00:00
Collecting scipy==1.12.0
Downloading scipy-1.12.0-cp310-cp310-macosx_12_0_arm64.whl (31.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.4/31.4 MB 1.2 MB/s eta 0:00:00
Collecting six==1.16.0
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting threadpoolctl==3.2.0
Downloading threadpoolctl-3.2.0-py3-none-any.whl (15 kB)
Collecting tzdata==2023.4
Downloading tzdata-2023.4-py2.py3-none-any.whl (346 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 346.6/346.6 kB 2.7 MB/s eta 0:00:00
Installing collected packages: pytz, tzdata, threadpoolctl, six, numpy, joblib, scipy, python-dateutil, scikit
-learn, pandas
Successfully installed joblib-1.3.2 numpy-1.26.3 pandas-2.2.0 python-dateutil-2.8.2 pytz-2023.3.post1 scikit-learn-1.5.0 scipy-1.12.0 six-1.16.0 threadpoolctl-3.2.0 tzdata-2023.4
I’m not an expert on Python package issues, but I suspect that there is a problem with the package versions and their compatibility with the specific architectures. What puzzles me is that the script runs successfully when executed directly in the terminal but fails when run by the Observable preview. What could be the cause?
I would greatly appreciate any help, hypotheses, or ideas!
Thanks!