BIDS Format Support and DuckDB Backend

Introduction

PhysioIO provides two complementary features for managing collections of physiological recordings:

BIDS support – read and write data in the Brain Imaging Data Structure (BIDS) format, the community standard for organizing neuroimaging and electrophysiology datasets.
DuckDB backend – store experiment metadata, channel information, events, and (optionally) signal data in an embedded analytical database for fast querying across many sessions.

This vignette demonstrates both capabilities and shows how they work together.

BIDS format support

What is BIDS?

The Brain Imaging Data Structure (BIDS) is a standard for organizing neuroimaging data into a consistent directory layout with machine-readable metadata files. PhysioIO supports the BIDS-EEG and BIDS-iEEG extensions.

A minimal BIDS dataset looks like this:

my_dataset/
  dataset_description.json
  sub-01/
    eeg/
      sub-01_task-rest_eeg.edf
      sub-01_task-rest_eeg.json
      sub-01_task-rest_channels.tsv
      sub-01_task-rest_events.tsv
      sub-01_task-rest_electrodes.tsv

Reference: Gorgolewski KJ, et al. (2016). “The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments.” Scientific Data, 3, 160044.

Reading from a BIDS dataset

library(PhysioIO)

# Read a single recording
pe <- readBIDS("path/to/bids", subject = "01", task = "rest")

# With session and run
pe <- readBIDS(
  "path/to/bids",
  subject = "01",
  session = "baseline",
  task = "oddball",
  run = 1,
  modality = "eeg"
)

# Skip loading events
pe <- readBIDS("path/to/bids", subject = "02", task = "rest",
               load_events = FALSE)

readBIDS() automatically loads the EDF data file and merges information from the companion _channels.tsv, _events.tsv, and _electrodes.tsv sidecar files when they are present.

Writing to BIDS format

# Export a PhysioExperiment to BIDS layout
writeBIDS(pe, "output/bids", subject = "01", task = "rest")

# With session and run identifiers
writeBIDS(pe, "output/bids",
          subject = "01", session = "baseline",
          task = "oddball", run = 1)

writeBIDS() creates the required directory structure, writes the EDF data file, and generates the sidecar TSV and JSON files.

Exploring a BIDS dataset

# List all subjects
subjects <- listBIDSSubjects("path/to/bids")
print(subjects)

# List sessions for a subject
sessions <- listBIDSSessions("path/to/bids", subject = "01")
print(sessions)

Validating BIDS compliance

result <- validateBIDS("path/to/bids")

if (result$valid) {
  message("Dataset is BIDS-compliant")
  message("Subjects found: ", result$n_subjects)
} else {
  message("Errors: ", paste(result$errors, collapse = "; "))
  message("Warnings: ", paste(result$warnings, collapse = "; "))
}

The validator checks for dataset_description.json, subject directory naming, modality directories, and required metadata fields.

DuckDB database backend

Why use a database?

When a study involves dozens or hundreds of recording sessions, it becomes impractical to load every file just to find which subjects had a particular task or which recordings fall within a date range. The DuckDB backend lets you register experiment metadata once and query it efficiently.

Reference: Raasveldt M, Muehleisen H (2019). “DuckDB: an embeddable analytical database.” Proceedings of the 2019 International Conference on Management of Data (SIGMOD).

Connecting and initializing

# In-memory database (useful for testing)
con <- connectDatabase()

# File-based database (persistent)
con <- connectDatabase("my_study.duckdb")

# Create the schema tables
initPhysioSchema(con)

Database schema

initPhysioSchema() creates the following tables:

Table	Purpose
`experiments`	One row per recording session (subject, task, sampling rate, etc.)
`channels`	Channel metadata (label, type, unit, electrode position)
`events`	Experimental events (onset, duration, type)
`signal_chunks`	Optional chunked signal data stored as BLOBs
`epochs`	Epoched data metadata
`annotations`	Free-form annotations

Registering experiments

# Read a recording
pe <- readEDF("sub01_rest.edf")

# Register metadata only
exp_id <- registerExperiment(con, pe,
  subject_id = "sub-01",
  task = "rest"
)

# Register with signal data stored in the database
exp_id <- registerExperiment(con, pe,
  subject_id = "sub-01",
  task = "rest",
  store_signals = TRUE,
  chunk_size = 10000L
)

Querying experiments

# All experiments
all_exps <- queryExperiments(con)

# Filter by subject
sub01 <- queryExperiments(con, subject_id = "sub-01")

# Filter by task
rest <- queryExperiments(con, task = "rest")

# Filter by date range
recent <- queryExperiments(con,
  date_range = c("2025-01-01", "2025-12-31")
)

Loading experiments from the database

# Load metadata (signal matrix is NA-filled)
pe <- loadExperiment(con, exp_id)

# Load with signal data
pe <- loadExperiment(con, exp_id, load_signals = TRUE)

Managing experiments

# Get database statistics
stats <- dbStats(con)
cat("Experiments:", stats$n_experiments, "\n")
cat("Channels:", stats$n_channels, "\n")
cat("Events:", stats$n_events, "\n")
cat("Subjects:", paste(stats$subjects, collapse = ", "), "\n")

# Delete an experiment and all related records
deleteExperiment(con, exp_id)

Disconnecting

# Always disconnect when done
disconnectDatabase(con)

Combining BIDS and DuckDB

A common workflow is to organize raw data in BIDS format on disk and register each recording in a DuckDB database for fast metadata queries.

# Set up database
con <- connectDatabase("study.duckdb")
initPhysioSchema(con)

# Iterate over BIDS subjects
bids_root <- "path/to/bids"
subjects <- listBIDSSubjects(bids_root)

for (subj in subjects) {
  # Read from BIDS

  pe <- readBIDS(bids_root, subject = subj, task = "rest")

  # Register in database
  registerExperiment(con, pe,
    subject_id = paste0("sub-", subj),
    task = "rest"
  )
}

# Now query across all subjects
all_rest <- queryExperiments(con, task = "rest")
print(all_rest)

disconnectDatabase(con)

Session info

sessionInfo()