13 Template Maintenance Playbook
This guide explains how to keep project repositories in sync with the Reproducible fMRI template.
13.1 Overview
┌─────────────────────┐ ┌─────────────────────┐
│ Template Repo │ │ Project Repo │
│ (Reproducible-fMRI) │ ──────▶ │ (your-study) │
│ │ │ │
│ • libs/paths.py │ sync │ • libs/paths.py │
│ • fMRIPrep scripts │ ──────▶ │ • fMRIPrep scripts │
│ • config/*.example │ │ • config/*.example │
│ • docs/ │ │ • docs/ │
│ │ │ │
│ │ ◀────── │ • analyses/ │
│ │ share │ • taskdata/ │
│ │ back │ • project docs │
└─────────────────────┘ └─────────────────────┘
Key principle: Infrastructure syncs both ways, but project-specific content stays local.
13.2 What Syncs vs. What Stays Local
The template defines three sync categories (enforced by sync_from_template.sh):
13.2.1 1. SAFE_INFRA — synced wholesale by default
Pure libraries with stable APIs, config templates, deploy helpers. Low customization risk; rare for child repos to fork these.
| Files | Why |
|---|---|
libs/bids_statsmodels.py |
BIDS Stats Model validation & generation |
libs/confounds.py |
Confound loading with task-type presets |
libs/reporting/* |
Generic HTML/PPTX report generator |
config/*.example.toml |
Configuration templates (you edit your own .toml) |
scripts/deploy/sync_pipeline_scripts.sh |
Deployment helper |
setup.sh, setup.py |
Setup wizard |
13.2.2 2. SYNC_WITH_CARE — skipped by default, opt-in flag required
These files are frequently customized per child repo. Blanket-syncing them would clobber legitimate project-specific work. Use opt-in flags or git checkout template/main -- <file> after manual review.
| Files | Common customizations | Opt-in flag |
|---|---|---|
libs/paths.py |
Custom dataclass fields, project-specific env var names, fallback parsers | --include-paths |
preprocessing/fmri/run_fmriprep_*.sh |
FMRIPREP_LAYOUT, PERSISTENT_WORK_ROOT, batch-specific derivatives |
--include-shells |
preprocessing/fmri/run_mriqc_*.sh, run_xcpd_*.sh, run_glmsingle_*.sh |
Per-tool tuning, resource overrides | --include-shells |
analyses/fmri/run_fitlins_*.sh |
Project-specific BIDS Stats Model invocations | --include-shells |
13.2.3 3. NEVER_SYNCS — project-specific, never touched
| Files | Why |
|---|---|
config/paths.toml |
Your data locations |
config/tooling.toml |
Your API tokens |
analyses/ (except run_fitlins_*.sh in #2) |
Your analysis code |
taskdata/ |
Your behavioral data processing |
experiments/ |
Your experiment code |
Project-specific docs (e.g. RESULTS.md, EXPERIMENTS.md) |
Your study documentation |
13.3 Customization Patterns: Extend, Don’t Fork
The cardinal rule: never edit a SAFE_INFRA or SYNC_WITH_CARE file in a child repo. Use the template’s documented extension mechanisms instead. Forks silently rot — every future template improvement requires manual merge work and regression risk.
13.3.1 Pattern 1 — Project-specific paths via [paths.locations]
Wrong (forking): add a new dataclass field to libs/paths.py:
# DON'T DO THIS — your child's paths.py will diverge from template forever
@dataclass(frozen=True)
class PathConfig:
...
prf_atlas_root: Path # ← project-specific additionRight (extending): declare it in config/paths.toml and access via location():
# config/paths.toml — child-specific, never synced
[paths.locations]
prf_atlas_root = "dataset::derivatives/prf_atlas"
glmsingle_root = "dataset::derivatives/glmsingle"# In your analysis code
from libs.paths import get_paths
paths = get_paths()
atlas = paths.location("prf_atlas_root") # works without forking paths.pyThis is the convergence path for any child that previously forked libs/paths.py to add custom fields. Migrate the fields to [paths.locations], swap callers from paths.prf_atlas_root to paths.location("prf_atlas_root"), and you can take template libs/paths.py updates wholesale forever.
13.3.2 Pattern 2 — Project-specific env vars in shell scripts
If you need a project-specific knob in run_fmriprep_hpc.sh (e.g. FMRIPREP_LAYOUT=batch), the convergence options in priority order:
Use an existing env var. The template’s scripts already honor
BATCH_LABEL,TASK_FILTER,CIFTI_OUTPUT,PERSISTENT_WORK_ROOT(when restored),KEEP_WORK_DIR(when restored), andFMRIPREP_EXTRA_ARGS. Many “customizations” can be expressed via these.Wrap the template script in a child-specific launcher (
scripts/launchers/run_fmriprep_my_study.sh) that sets your env vars then exec’s the template’srun_fmriprep_hpc.sh. This keeps the sync-able script untouched.Upstream the feature. If your knob is generic enough (e.g.
FMRIPREP_LAYOUTdistinguishing batch vs. global derivatives is broadly useful), open a PR against the template instead of forking the script.Fork as last resort. If you must edit the script in place, document why in
KNOWN_ISSUES.mdand exclude the file from sync via./sync_from_template.sh --exclude preprocessing/fmri/run_fmriprep_hpc.sh.
13.3.3 Pattern 3 — Project-specific docs
Each child should have its own RESULTS.md, EXPERIMENTS.md, and any study-specific guides under docs/. The template’s DOC_FILES list (GETTING_STARTED.md, DATA_SETUP.md, HPC_GUIDE.md, ANALYSIS.md, KNOWN_ISSUES.md, DOCUMENTATION_INDEX.md) is opinionated and project- agnostic. Children with consolidated doc sets that overlap names but with project-specific content should --exclude the conflicting file from sync.
13.4 Convergence Playbook: Fixing a Diverged Repo
If a child has accumulated customizations and falls behind the template, work through this order. Each step is independent — pick the ones that apply.
Audit the drift. From the child repo:
./scripts/deploy/sync_from_template.sh --preview --diffThe
--diffflag shows actual content diffs per file. Categorize each: stale, missing, legitimate (extensible), legitimate (forked), conflicting. See the table in TEMPLATE_MAINTENANCE.md § “Divergence Taxonomy” below.Migrate “legitimate (extensible)” cases first. This is the highest-ROI work. Convert custom
paths.pydataclass fields to[paths.locations]entries. Update callers. Nowlibs/paths.pycan be safely synced from template.Apply SAFE_INFRA updates. Run without opt-in flags:
./scripts/deploy/sync_from_template.sh --infraThis brings in new utilities, configs, setup wizards.
Selectively apply SYNC_WITH_CARE updates. For each shell script you want to take wholesale (because you have no local edits, or you’ve migrated your knobs to env vars / launchers):
git checkout template/main -- preprocessing/fmri/run_fmriprep_hpc.sh git diff --staged # review carefullyDocument remaining divergence. For files you’ve intentionally forked, add an entry to
docs/KNOWN_ISSUES.md:### Diverged from template - `preprocessing/fmri/run_fmriprep_hpc.sh` — kept project's `FMRIPREP_LAYOUT` logic; not in template (PR #XX upstream)Add CI guard. Add a CI job that runs
sync_from_template.sh --previewand fails if the SAFE_INFRA category has unmerged updates older than N weeks. Prevents drift from re-accumulating.
13.5 Divergence Taxonomy
When auditing drift, classify each file:
| Type | Meaning | Convergence path |
|---|---|---|
| Stale | Child’s version is just an old snapshot; no project intent diverged | Sync wholesale |
| Missing | Template added new file/feature; child doesn’t have it | Sync wholesale (additive) |
| Legitimate (extensible) | Child has project-specific extension; template’s API supports it via existing pattern | Migrate to extension pattern (Pattern 1/2 above), then sync |
| Legitimate (forked) | Child needs feature template doesn’t support; legitimate fork | Keep child’s; document in KNOWN_ISSUES.md; consider upstreaming |
| Conflicting | Both evolved differently for the same feature | Manual merge with documented rationale |
13.6 Quick Sync: Pull Template Updates
13.6.1 Option 1: Using the Helper Script (Recommended)
# From your project repository
# 1. Preview safe-infra changes (no SYNC_WITH_CARE files shown for action)
./scripts/deploy/sync_from_template.sh --preview
# 2. Preview with actual content diffs (slower, more informative)
./scripts/deploy/sync_from_template.sh --preview --diff
# 3. Apply safe-infra + docs (default mode)
./scripts/deploy/sync_from_template.sh
# 4. Apply only safe-infra (skip docs)
./scripts/deploy/sync_from_template.sh --infra
# 5. Opt in to syncing libs/paths.py (only if you have NO custom dataclass fields)
./scripts/deploy/sync_from_template.sh --include-paths
# 6. Opt in to syncing preprocessing/*.sh shell scripts (only if you have no
# project-specific env vars in them)
./scripts/deploy/sync_from_template.sh --include-shells
# 7. Skip specific files even within the safe set (e.g. you've forked setup.sh)
./scripts/deploy/sync_from_template.sh --exclude setup.sh,libs/reporting/__init__.py13.6.2 Option 2: Manual Git Merge
# Add template remote (one-time)
git remote add template https://github.com/CNClaboratory/Reproducible-fMRI.git
# Fetch and merge
git fetch template
git merge template/main --no-commit --allow-unrelated-histories
# Resolve conflicts (keep your project-specific files)
git checkout --ours config/paths.toml # Keep your config
git checkout --theirs libs/paths.py # Take template's version
# Review and commit
git diff --staged
git commit -m "Sync infrastructure from template"13.6.3 Option 3: Cherry-Pick Specific Changes
git fetch template
git log template/main --oneline -10 # Find commit to cherry-pick
git cherry-pick <commit-hash>13.7 Push Improvements Back to Template
When you develop something reusable in a project:
13.7.1 1. Identify Reusable Work
Ask: “Would another project benefit from this?”
✅ Good candidates: - Bug fixes to libs/paths.py - New fMRIPrep options in preprocessing scripts - Documentation improvements - New utility scripts
❌ Keep local: - Project-specific analysis code - Dataset-specific preprocessing - Study-specific documentation
13.7.2 2. Isolate the Change
Ensure it’s generic: - No hardcoded paths (use config/paths.toml) - No project-specific variable names - Include .example config files if needed
13.7.3 3. Create PR to Template
# From the template repo
git fetch origin
git checkout -b feature/my-improvement origin/main
# Copy your changes
cp /path/to/project/libs/paths.py libs/paths.py
# Or cherry-pick if it's a clean commit
git remote add project /path/to/project
git fetch project
git cherry-pick <commit>
# Push and create PR
git push -u origin feature/my-improvement13.8 Sync Schedule
| When | Action |
|---|---|
| Before new study | Sync template to get latest best practices |
| Monthly | Quick check for template updates |
| After major milestone | Review project for reusable improvements |
13.9 Conflict Resolution
When merging template updates, conflicts may arise. Resolution strategy:
| File Type | Strategy |
|---|---|
config/paths.toml |
Always keep yours (--ours) |
libs/paths.py |
Default: keep yours. Only take template’s (--theirs) after migrating any custom dataclass fields to [paths.locations] in your paths.toml (see Pattern 1 above). The --include-paths flag is opt-in for this reason. |
preprocessing/*.sh |
Default: keep yours if you have project-specific env vars (FMRIPREP_LAYOUT, etc.). Take template’s via --include-shells only after wrapping your knobs in a launcher script (see Pattern 2). |
docs/ (canonical 6) |
Take template’s wholesale unless you have project-specific content under the same filename. In that case, --exclude docs/<file>.md and merge selectively. |
Project-specific docs (RESULTS.md, EXPERIMENTS.md) |
Always keep yours — these are not in DOC_FILES. |
13.10 Checking Sync Status
# See what's different between your project and template
git fetch template
git diff HEAD template/main -- libs/paths.py
git diff HEAD template/main -- preprocessing/fmri/
# List files that differ
git diff --name-only HEAD template/main13.11 Troubleshooting
13.11.2 Template remote not working
# Verify remote exists
git remote -v
# Re-add if needed
git remote remove template
git remote add template https://github.com/CNClaboratory/Reproducible-fMRI.git13.11.3 Merge conflicts overwhelming
Use the infrastructure-only sync (skips docs):
./scripts/deploy/sync_from_template.sh --infraThis only syncs SAFE_INFRA files; SYNC_WITH_CARE files (paths.py, shell scripts) and documentation are skipped. Use --diff to see exact content changes before applying:
./scripts/deploy/sync_from_template.sh --preview --diff --infraIf a specific safe-infra file has conflicts you don’t want, exclude it:
./scripts/deploy/sync_from_template.sh --infra --exclude setup.sh13.12 Child Repo Sync Guide (Feb 2026)
After the BIDS Stats Models integration and pipeline audit, the following new files are available in the template. Each child repo should sync selectively based on its needs.
13.12.1 New Template Files Available
| File | Category | Pull Into Child? |
|---|---|---|
libs/bids_statsmodels.py |
Python utility | Yes - model validation and generation |
libs/confounds.py |
Python utility | Yes - standardized confound loading |
config/glm_defaults.example.toml |
Config template | Yes - copy and customize |
preprocessing/fmri/run_mriqc_batch.sh |
MRIQC launcher | Yes (if not already customized) |
preprocessing/fmri/run_mriqc_hpc.sh |
MRIQC launcher | Yes (if not already customized) |
preprocessing/fmri/run_xcpd_batch.sh |
XCP-D launcher | Yes (if not already customized) |
preprocessing/fmri/run_xcpd_hpc.sh |
XCP-D launcher | Yes (if not already customized) |
preprocessing/fmri/run_glmsingle_batch.sh |
GLMsingle launcher | Yes - new |
preprocessing/fmri/run_glmsingle_hpc.sh |
GLMsingle launcher | Yes - new |
analyses/fmri/run_fitlins_batch.sh |
FitLins launcher | Optional - for BIDS Stats Models execution |
analyses/fmri/run_fitlins_hpc.sh |
FitLins launcher | Optional - for BIDS Stats Models execution |
analyses/fmri/models/*.smdl.json |
Example models | Copy and customize for your study |
scripts/deploy/sync_pipeline_scripts.sh |
Deploy helper | Yes - replaces sync_fmriprep_scripts.sh |
13.12.2 Per-Repo Instructions
13.12.2.1 twcf_figureground_fmri_analysis
This repo already has mature MRIQC, XCP-D, and GLM scripts.
# Pull only NEW utilities (don't overwrite existing pipeline scripts)
git fetch template
git checkout template/main -- libs/bids_statsmodels.py
git checkout template/main -- libs/confounds.py
git checkout template/main -- config/glm_defaults.example.toml
git checkout template/main -- preprocessing/fmri/run_glmsingle_batch.sh
git checkout template/main -- preprocessing/fmri/run_glmsingle_hpc.sh
git checkout template/main -- scripts/deploy/sync_pipeline_scripts.sh
# Optional: copy example models as starting points
git checkout template/main -- analyses/fmri/models/
# DO NOT overwrite (already customized):
# - preprocessing/fmri/run_mriqc_batch.sh
# - preprocessing/fmri/run_mriqc_hpc.sh
# - preprocessing/fmri/run_xcpd_batch.sh
# - preprocessing/fmri/run_xcpd_hpc.sh
# - config/glm_defaults.toml (already has project-specific settings)
# - analyses/fmri/glm/ (project-specific GLM code)
# Config: Add new pipeline locations to paths.toml
# mriqc_root = "dataset::derivatives/mriqc"
# xcpd_root = "dataset::derivatives/xcpd"
# glmsingle_root = "dataset::derivatives/glmsingle"13.12.2.2 vividness-perception-imagery
This repo has the 3-branch architecture already documented.
# Pull new utilities
git fetch template
git checkout template/main -- libs/bids_statsmodels.py
git checkout template/main -- libs/confounds.py
git checkout template/main -- config/glm_defaults.example.toml
git checkout template/main -- preprocessing/fmri/run_glmsingle_batch.sh
git checkout template/main -- preprocessing/fmri/run_glmsingle_hpc.sh
# Pull generalized MRIQC/XCP-D scripts (if not already customized)
git checkout template/main -- preprocessing/fmri/run_mriqc_batch.sh
git checkout template/main -- preprocessing/fmri/run_mriqc_hpc.sh
# Optional: FitLins (if planning to use BIDS Stats Models)
git checkout template/main -- analyses/fmri/run_fitlins_batch.sh
git checkout template/main -- analyses/fmri/run_fitlins_hpc.sh
git checkout template/main -- analyses/fmri/models/
# DO NOT overwrite:
# - analyses/fmri/glm_first_level.py (project-specific 24P strategy)
# - preprocessing/fmri/run_xcpd_*.sh (if already configured for linc mode)13.12.2.3 TI_DecNef
This repo has a MATLAB/Python hybrid setup. Sync cautiously.
# Template remote already added. Pull new utilities:
git fetch template
git checkout template/main -- libs/bids_statsmodels.py
git checkout template/main -- libs/confounds.py
git checkout template/main -- config/glm_defaults.example.toml
# Pull pipeline launchers (all new for this repo)
git checkout template/main -- preprocessing/fmri/run_mriqc_batch.sh
git checkout template/main -- preprocessing/fmri/run_mriqc_hpc.sh
git checkout template/main -- preprocessing/fmri/run_xcpd_batch.sh
git checkout template/main -- preprocessing/fmri/run_xcpd_hpc.sh
git checkout template/main -- preprocessing/fmri/run_glmsingle_batch.sh
git checkout template/main -- preprocessing/fmri/run_glmsingle_hpc.sh
git checkout template/main -- scripts/deploy/sync_pipeline_scripts.sh
# DO NOT overwrite:
# - MATLAB analysis code (experiments/, analyses/decnef/)
# - Project-specific preprocessing that interfaces with MATLAB13.12.3 Configuration Changes for All Child Repos
After syncing, each repo should:
Copy config template:
cp config/glm_defaults.example.toml config/glm_defaults.tomlEdit confound strategy: Choose presets appropriate for your analysis type
Add pipeline locations to
config/paths.toml(if using new pipelines):[paths.locations] mriqc_root = "dataset::derivatives/mriqc" xcpd_root = "dataset::derivatives/xcpd" glmsingle_root = "dataset::derivatives/glmsingle" fitlins_root = "dataset::derivatives/fitlins"Ensure
pyproject.tomlhasjsonschema:uv add jsonschemaRun tests:
uv run python -m pytest tests/test_bids_statsmodels.py tests/test_confounds.py -v
13.13 Per-Child Convergence Roadmap (April 2026 audit)
Audit run on 2026-04-26 against template main after the docs consolidation + sync-categorization landing. Each child was scored against the divergence taxonomy (see § “Divergence Taxonomy” above). Use this section to drive the next convergence pass; update entries as work lands.
13.13.1 Summary
| Child | State | Highest-ROI action |
|---|---|---|
| twcf | Heavy fork of libs/paths.py (project-specific dataclass fields); custom env vars in run_fmriprep_hpc.sh (FMRIPREP_LAYOUT, PERSISTENT_WORK_ROOT). Mature consolidated doc set already in place. |
Migrate paths.py custom fields to [paths.locations]; wrap run_fmriprep_hpc.sh knobs in a launcher; document remaining forks in KNOWN_ISSUES.md. |
| vividness | Multi-site (UCI + NEU). Has three improvements over template (optional BATCH_LABEL, datalad_epilog trap, 128 G XCP-D memory). Several *-uci.sh site-specific forks. Custom stimuli-modality fields in paths.py. |
Upstream the three improvements to template, then migrate paths.py stimuli fields to [paths.locations], then sync. |
| Hypergraphsciousness | Uses NKI preprocessing — fMRIPrep scripts are operationally dormant. All divergences are stale snapshots, not intentional forks. xgi/nilearn conflict properly declared via pyproject.toml. |
Low priority — optional additive sync of libs/bids_statsmodels.py, libs/confounds.py, libs/reporting/*. fMRIPrep script sync is cosmetic. |
| TI_DecNef | Manually synced 2026-04-07 (libs/confounds.py, libs/bids_statsmodels.py, libs/guardrail_log.py are current). No sync script installed. Intentional fork of preprocessing/fmri/*.sh for UCI HPC3 specifics. MATLAB/Python hybrid. |
Optional safe additives: config/glm_defaults.example.toml, config/tooling.example.toml. Do not install sync_from_template.sh blindly; do not blanket-sync preprocessing. |
13.13.2 Numpy version coordination across child repos
Canonical floor (recommended): numpy>=2.0. This aligns with the template’s implicit baseline (via current nilearn>=0.11, xgi) and matches three of four child repos.
| Repo | Current pin | Status |
|---|---|---|
| Reproducible-fMRI (template) | (no explicit pin; transitive ≥ 2.0 via nilearn/xgi) | Canonical |
| vividness | numpy>=2.3.3 |
Compatible |
| Hypergraphsciousness | numpy>=2.0.2 |
Compatible |
| TI_DecNef | numpy>=1.24.0 |
Compatible (floats; accepts v1 or v2) |
| twcf | numpy>=1.23,<2.0 |
Diverged — pinned to v1 ceiling |
Why twcf is on v1: at the time of pinning, an upstream dep (likely an older nilearn or scipy chain) still required v1. As of April 2026, this constraint is no longer needed; the pin survives because shipping the CCN 2026 manuscript took priority over dep upgrades.
Code-level impact: shared library code (libs/) is currently safe — the audit found no np.cumproduct, np.product, np.alltrue, np.NaN, or np.fromstring usages (the v1-only patterns removed in v2). Future contributors should keep this discipline so cross-repo helpers don’t break twcf.
Convergence path (when twcf is post-manuscript): 1. Bump twcf to numpy>=2.0 in pyproject.toml. 2. Run uv lock to refresh uv.lock. 3. Run twcf’s full test suite + smoke pipeline on real data. 4. Open follow-up issues for any v1-only patterns surfaced in twcf-specific code.
Until then, write shared library code that is v1+v2 compatible (avoid the removed APIs above; prefer modern aliases like numpy.bool_ over deprecated numpy.bool).
13.13.3 Upstream candidates (template-side improvements from child observations)
These are features children built that should go INTO the template:
- Optional
BATCH_LABELinrun_fmriprep_hpc.sh,run_xcpd_hpc.sh,run_glmsingle_hpc.sh— vividness moved to standard BIDS layout (derivatives/<tool>/) by default; legacy batch-subdir only whenBATCH_LABELis set. Backward-compatible improvement. datalad_epilogtrap inrun_xcpd_hpc.sh,run_glmsingle_hpc.sh— vividness adds auto-save + metadata push on job exit. Production-tested.- 128 G memory floor for XCP-D 0.10+ with full atlas parcellation + CIFTI — vividness fixes OOM at 64 G. The current template default needs a bump or per-mode resource selection.
Track these as separate template-side issues; do NOT entangle with child convergence work.
13.13.4 Per-child handoff prompts
Each prompt is meant to be copy-pasted into a fresh Claude session opened in that child repo. It carries forward the audit context so the agent doesn’t have to redo discovery.
13.13.4.1 Handoff: twcf_figureground_fmri_analysis
You are working in the twcf_figureground_fmri_analysis child of the
Reproducible-fMRI template. Your task: converge this repo to the template's
canonical patterns where safe, document where divergence is legitimate, and
make future sync_from_template.sh runs safer.
CONTEXT FROM 2026-04-26 AUDIT (template-side):
- Template now has SAFE_INFRA / SYNC_WITH_CARE / NEVER_SYNCS categories.
See template's docs/TEMPLATE_MAINTENANCE.md § "Customization Patterns".
- This repo has heavy customization in two SYNC_WITH_CARE files:
1. libs/paths.py — custom dataclass fields (prf_atlas_root,
glmsingle_root, results_cache_root, xcpd_root, prf_fitted_root),
custom env var name TWCFFG_PATHS_FILE, fallback TOML parser for HPC
login nodes.
2. preprocessing/fmri/run_fmriprep_hpc.sh — uses FMRIPREP_LAYOUT
(batch vs global derivatives), PERSISTENT_WORK_ROOT, KEEP_WORK_DIR.
- This repo has its own consolidated 10-doc set already; do NOT sync the
template's docs/ wholesale. Only consider per-file merges.
YOUR TASKS (in priority order):
1. Migrate libs/paths.py custom fields to [paths.locations] entries:
- Read the template's libs/paths.py to confirm `paths.location("name")`
is the canonical extension API.
- For each custom field (prf_atlas_root, glmsingle_root, etc.),
add an entry to config/paths.toml's [paths.locations] section.
- Update all callers in this repo: `paths.prf_atlas_root` ->
`paths.location("prf_atlas_root")`. Use grep to find them all.
- Run unit tests + a smoke pipeline run to verify nothing broke.
- Commit as a single migration: "refactor(paths): migrate custom
fields to paths.locations for template convergence".
- After this lands, this repo can take template's libs/paths.py
wholesale. Sync it: `./scripts/deploy/sync_from_template.sh
--include-paths --preview` then apply.
2. Wrap run_fmriprep_hpc.sh customizations in a launcher:
- Create scripts/launchers/run_fmriprep_twcf.sh that sets
FMRIPREP_LAYOUT, PERSISTENT_WORK_ROOT, KEEP_WORK_DIR env vars then
execs the template's run_fmriprep_hpc.sh.
- Update Makefile / docs to invoke the launcher, not the template
script directly.
- Now run_fmriprep_hpc.sh is sync-safe.
3. Apply SAFE_INFRA sync:
- `./scripts/deploy/sync_from_template.sh --infra --diff`
- Review carefully (config/paths.example.toml restructure is largest
change). Their real config/paths.toml is gitignored, so this is
just an example update.
- Commit.
4. Document remaining forks in docs/KNOWN_ISSUES.md under a new section
"Diverged from template":
- List each file you intentionally kept forked.
- Brief reason and link to upstream PR if applicable.
5. Update twcf's local sync_from_template.sh to match the new template
version (with --include-paths, --include-shells, --exclude flags).
CONSTRAINTS:
- This repo is actively shipping CCN 2026 manuscript work. Do NOT break
the pipeline. Run tests/smoke before committing.
- Their consolidated 10-doc set must not be clobbered.
- Their AGENTS.md has strict code placement rules — read it first.
Investigate before each step. Ask before any destructive change.
13.13.4.2 Handoff: vividness-perception-imagery
You are working in the vividness-perception-imagery child of the
Reproducible-fMRI template. Your task: capture and upstream the three
improvements this repo developed, then converge.
CONTEXT FROM 2026-04-26 AUDIT (template-side):
- This repo is AHEAD of the template on three features:
1. Optional BATCH_LABEL in run_fmriprep_hpc.sh, run_xcpd_hpc.sh,
run_glmsingle_hpc.sh — defaults to standard BIDS layout
(derivatives/<tool>/), batch-subdir only when set.
2. datalad_epilog trap in run_xcpd_hpc.sh, run_glmsingle_hpc.sh —
auto-save + metadata push on exit.
3. 128G memory floor for XCP-D 0.10+ with CIFTI (template has 64G
which OOMs).
- Template now has SAFE_INFRA / SYNC_WITH_CARE / NEVER_SYNCS categories.
- This repo has multi-site forks: run_*-uci.sh variants for HPC3 specifics.
These are LEGITIMATE — keep them, document the pattern.
- Custom fields in libs/paths.py: 4 stimuli modalities, analysis_derivatives_root,
fmriprep_container, stats_models_root.
YOUR TASKS (in priority order):
1. Upstream the three improvements to the template:
- Open template at /home/yoursurname/src/github.com/CNClaboratory/Reproducible-fMRI
- For each of the 3 features, prepare a clean PR-style commit
against the template's main:
a. Optional BATCH_LABEL in 3 fmriprep/xcpd/glmsingle hpc scripts
b. datalad_epilog trap pattern in xcpd/glmsingle hpc scripts
c. XCP-D memory bump (or per-mode logic: 128G for CIFTI/full
atlas, 64G for minimal)
- Each as a separate commit with a clear message and rationale.
- Push to template main (or open a PR if you prefer review).
2. After upstreaming, sync this repo's preprocessing scripts:
- `./scripts/deploy/sync_from_template.sh --include-shells --diff`
- Now the template has your features; the sync becomes a no-op
for those files. Other shell-script updates can be taken cleanly.
3. Migrate libs/paths.py extra fields to [paths.locations]:
- 4 stimuli modalities (lab/fmri/online/generic) -> stimuli_lab_root,
stimuli_fmri_root, etc. as locations
- analysis_derivatives_root, fmriprep_container, stats_models_root
-> locations
- Update callers (paths.X -> paths.location("X"))
- Then sync libs/paths.py: `./scripts/deploy/sync_from_template.sh
--include-paths`
4. Document UCI-specific *-uci.sh forks:
- Add docs/SITE_SPECIFIC.md (or similar) explaining the pattern
- Each *-uci.sh is a thin wrapper that sets UCI-specific env then
execs the generic script — verify this is the architecture; if
not, refactor toward it.
5. Apply SAFE_INFRA sync.
6. Verify NEU pilot (Michaela's stress test) still works after each
convergence step. Do not break multi-site.
CONSTRAINTS:
- Multi-site reproducibility is the explicit goal. Test on both UCI
and NEU pathways (or at least dry-run on both).
- This repo is actively shipping ETHOS pilot. Smoke before commit.
Investigate before each step. Ask before pushing to template.
13.13.4.3 Handoff: Hypergraphsciousness
You are working in the Hypergraphsciousness child of the
Reproducible-fMRI template. Your task is LIGHT — this repo's divergence
is mostly stale snapshots, and many template features (fmriprep scripts)
are operationally dormant here because you use NKI preprocessing.
CONTEXT FROM 2026-04-26 AUDIT (template-side):
- This repo uses NATVIEW NKI preprocessed derivatives, NOT fMRIPrep.
- All preprocessing/fmri/*.sh divergences are stale snapshots, not
intentional forks — but they're also operationally dormant.
- Template now has SAFE_INFRA / SYNC_WITH_CARE / NEVER_SYNCS categories.
- xgi/nilearn dependency conflict is properly handled in pyproject.toml
via separate conflicts extras — no template change needed.
YOUR TASKS (priority order — most are optional):
1. Apply SAFE_INFRA sync (low effort, low risk):
- `./scripts/deploy/sync_from_template.sh --infra --diff`
- Brings in libs/reporting/* (if you ever want generic HTML reports),
updated configs, sync_pipeline_scripts.sh, setup.sh updates.
- Commit.
2. SKIP libs/paths.py sync unless you explicitly want template's
two-tier design. Your version works.
3. SKIP fmriprep/mriqc/xcpd shell script syncs — they're dormant.
Document this in docs/KNOWN_ISSUES.md "Diverged from template"
section: "Preprocessing/fmri scripts intentionally lag template;
this repo uses NKI preprocessing."
4. Update local sync_from_template.sh to the new template version
(with the new flags) by syncing it as part of step 1.
5. Decision point: if you're going to start using fMRIPrep at any
point, plan a formal sync of preprocessing scripts then. Until
then, no urgency.
CONSTRAINTS:
- Don't introduce new dependency conflicts. xgi/nilearn separation is
fragile; verify pyproject.toml after sync.
Quick task — should take under an hour.
13.13.4.4 Handoff: TI_DecNef
You are working in the TI_DecNef child of the Reproducible-fMRI template.
Your task is LIGHT — this repo was already manually synced on 2026-04-07
for core libs, and its preprocessing scripts are intentionally forked for
UCI HPC3 + the MATLAB/Python hybrid pipeline.
CONTEXT FROM 2026-04-26 AUDIT (template-side):
- libs/confounds.py, libs/bids_statsmodels.py, libs/guardrail_log.py
were synced via cherry-pick on 2026-04-07 (commit 5e1323b). Current.
- libs/paths.py is functionally equivalent to template (517 LOC vs 514).
Missing _LOCAL_CONFIG_FILE override pattern but works.
- preprocessing/fmri/*.sh are intentional forks for UCI HPC3 SLURM
specifics. Do NOT sync template versions blindly.
- No sync_from_template.sh installed. Manual cherry-picks have been
sufficient for this single child repo.
- Template now has SAFE_INFRA / SYNC_WITH_CARE / NEVER_SYNCS categories.
YOUR TASKS (priority order):
1. Optional safe additives via cherry-pick (no sync script needed):
git fetch template git checkout template/main – config/glm_defaults.example.toml git checkout template/main – config/tooling.example.toml git diff –staged git commit -m “feat(config): add glm_defaults + tooling examples from template” ```
- Optional: sync libs/paths.py to get _LOCAL_CONFIG_FILE pattern:
- First read template’s libs/paths.py changes
- This is additive (your dataclass is similar enough)
git checkout template/main -- libs/paths.pyif no custom fields- Verify imports + a smoke test before commit
- SKIP installing sync_from_template.sh:
- You have a single-child + manual cherry-pick workflow that works
- Premature automation risks breaking your intentional preprocessing forks
- Re-evaluate if you ever onboard a second researcher who needs automated sync
- SKIP setup.sh / setup.py:
- Your scripts/setup_hpc.sh is appropriate for HPC-primary workflow
- Template’s full wizard adds complexity you don’t need
- Document the sync strategy in docs/KNOWN_ISSUES.md:
- “TI_DecNef syncs from template via manual cherry-pick, not sync_from_template.sh, due to: (a) MATLAB/Python hybrid, (b) single-child status, (c) intentional preprocessing forks.”
CONSTRAINTS: - This repo is in manuscript-writing mode. Don’t introduce churn. - Don’t disturb the MATLAB→Python pipeline boundaries.
Quick task — should take under 30 minutes. ```
13.14 Contact
Questions about syncing? Open an issue in the template repository.