Github Actions with pandas data loader on Observable Framework

New To Observable
1

hello,

I am trying to implement Github Actions upon pushing my Observable Framework repo to the main branch (identical to the one on the official website, only without the scheduled run):

name: Deploy

on:
  # Run this workflow whenever a new commit is pushed to main.
  push: {branches: [main]}
  # Run this workflow when triggered manually in GitHub’s UI.
  workflow_dispatch: {}

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm run build
      - name: Deploy to Observable Cloud
        # This parameter to `--message` will use the latest commit message
        run: npm run deploy -- --message "$(git log -1 --pretty=%s)"
        env:
          # Authentication information. See below for how to set this up.
          OBSERVABLE_TOKEN: ${{ secrets.OBSERVABLE_TOKEN }}

My project includes a data loader (e.g., dataloader.csv.py) which contains a pandas import:

import pandas as pd

However, npm run build fails after encountering the following error:

Traceback (most recent call last):
  File "/home/runner/work/projects/src/data/dataloader.csv.py", line 3, in <module>
    import pandas as pd
ModuleNotFoundError: No module named 'pandas'
load src/data/dataloader.csv.py → [missing] error in 94ms: loader exited with code 1

Unexpected error: loader exited with code 1

As importing pandas works with my local computer, I guess the program refers to my local python environment somehow, which I haven’t understood yet.

How can I know the path to the python environment is used? Also, what would be the best practice for using a virtual environment with the Observable Framework project?

Thank you

2

Okay, here is a quick fix:

  1. Create a requirements.txt file in my repo:
pandas
  1. Update the GitHub Actions workflow to set up Python and install dependencies:
name: Deploy

on:
  push: {branches: [main]}
  workflow_dispatch: {}

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.x'
      - name: Install Python dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
      - run: npm ci
      - run: npm run build
      - name: Deploy to Observable Cloud
        run: npm run deploy -- --message "$(git log -1 --pretty=%s)"
        env:
          OBSERVABLE_TOKEN: ${{ secrets.OBSERVABLE_TOKEN }}
3

I’m running into a similar issue, and while the .yml file that @gshirato provided half-helped, it didn’t fully solve the problem. I’d really appreciate any guidance!

I was getting the same issue (but with ‘import requests’), and the deploy was failing. After creating the .yml file, I’m still getting the error message, and the build is reporting as a failure on GitHub, but deploy itself is successful, and the actual app page over on Observable Framework gets properly updated.

Here’s the full error message, in case it’s helpful:

18:22:27.999Z
build
-
copy src/data/nodegoat_data.json → load /data/nodegoat_data.json → [missing] stderr: Traceback (most recent call last):
18:22:28.015Z
build
-
  File "/project/src/data/nodegoat_data.json.py", line 5, in <module>
18:22:28.015Z
build
-
stderr:     import requests
18:22:28.015Z
build
-
ModuleNotFoundError: No module named 'requests'
18:22:28.018Z
build
-
error in 18ms: loader exited with code 1
18:22:28.018Z
build
-
stderr: 
18:22:28.018Z
build
-
Unexpected error: loader exited with code 1
18:22:28.018Z
build
-
stderr: The full error follows
18:22:28.018Z
build
-
18:22:28.019Z
build
-
stderr: file:///project/node_modules/@observablehq/framework/dist/loader.js:430
18:22:28.019Z
build
-
      throw new Error(`loader exited with code ${code}`);
18:22:28.019Z
build
-
            ^
18:22:28.019Z
build
-
18:22:28.019Z
build
-
Error: loader exited with code 1
18:22:28.019Z
build
-
    at CommandLoader.exec (file:///project/node_modules/@observablehq/framework/dist/loader.js:430:13)
18:22:28.019Z
build
-
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
18:22:28.019Z
build
-
    at async file:///project/node_modules/@observablehq/framework/dist/loader.js:373:11
18:22:28.019Z
build
-
18:22:28.019Z
build
-
Node.js v20.17.0
18:22:29.007Z
finalizing
-
Aborting build due to error
18:22:29.007Z
build
-
Error with build: Command failed exitCode: 1
4

I finally figured out how to solve the second problem, with reference to this page. I also needed to add a pip install line to my package.json file, so that it now reads:

# ...
"observable": "observable",
"install": "pip install -r requirements.txt"
#...