projects:biosonification:sfcs [wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Plant Touch Sonification — Project Reference ======

===== Goal =====

Touch-based interactive botanical installation for sight-impaired visitors. Visitors touch plants, plants produce sound. Must run for months without damaging plants. Signal must be meaningful, not random.

===== Approach: SFCS (Swept Frequency Capacitive Sensing) =====

Disney's Touché technique, reimplemented with open-source hardware. Single electrode in soil, no contact with plant tissue.

Traditional capacitive sensing reads conductivity at one fixed frequency; SFCS uses a whole range so a single sensor's output becomes two-dimensional.

Electrode goes in soil, not on the plant. No adhesive, no gel drying out, no girdling as stems grow. The months-long electrode degradation problem that kills passive-sensing approaches doesn't apply.

==== The sweep ====

  * Range: **1.5 kHz – 3.5 MHz** ([[https://github.com/chonigman/SweepingCapSense|library README]]). Disney's AD5932 covered 1 kHz–3.5 MHz with a true sine
  * **Up to 198 steps** per the paper. (library README says 200 max.)
  * //sample code does 100Khz to 16Mhz//
  * Waveform generated by AVR 16-bit hardware timers via direct register manipulation ([[https://github.com/Illutron/AdvancedTouchSensing|Møbius's original code]] from [[https://dzlsevilgeniuslair.blogspot.com/2012/05/arduino-do-touche-dance.html|this post]] uses hex/binary timer addressing — the library exists to wrap that)
  * Arduino can hit those frequencies but only as a square wave with lower resolution, producing unwanted harmonics. The LC resonant circuit reshapes it toward a sinusoid
  * Output per sweep is a value at every frequency, not just a peak — the //shape// of the curve is the signal


==== The sweep ====
 
  * Range: **1.5 kHz – 3.5 MHz** (library README). Disney's AD5932 covered 1 kHz–3.5 MHz with a true sine
  * **Up to 198 steps** per the paper; library README says 200 max. Unresolved discrepancy
  * Waveform generated by AVR **16-bit hardware timers** via direct register manipulation (Møbius's original code uses hex/binary timer addressing — the library exists to wrap that)
  * Arduino can hit those frequencies but only as a square wave with lower resolution, producing unwanted harmonics. The LC resonant circuit reshapes it toward a sinusoid
  * Output per sweep is a value at every frequency, not just a peak — the //shape// of the curve is the signal
 
==== Sweep mechanics (verified from SweepingCap.cpp) ====
 
  * The number you pass to ''sweep()'' is backwards from frequency. Bigger number = lower frequency. f = 16,000,000 / (1 + n).
  * Stepping that number 1, 2, 3, 4… does not give evenly spaced frequencies. It's badly lopsided — the first few steps jump megahertz, the last few move a few hundred Hz. For even coverage, precompute the values and pass those instead of a raw counter.
  * All four channels sweep together. You can't have plant A at one frequency while plant B is at another.

==== Repeatability ====

The NIME 2014 paper reports curve shapes repeat reliably even when the specific peak frequency and value drift. This matters for a long-running install — key on shape, not absolute peak.

Also position-sensitive, not just gesture-sensitive: the same two-finger pinch at different points on the rosemary (trunk, branch tip, etc.) produced distinguishable curves.

==== Scaling ceiling — important for multi-plant ====

Cultivating Frequencies ([[https://colinhonigman.com/Cultivating-Frequencies|project writeup]], [[https://github.com/chonigman/CultivatingFrequencies|Max/MSP patch]], [[https://www.cultivatingfrequencies.com|installation site]]) ran fifteen plants off the four 16-bit timers of one Mega. This demanded significant processing power and memory, and that the system slows noticeably with each added touch point, since every pin must be read at every frequency. Not reading all sensors simultaneously helps but constrains interaction.

//Fifteen plants on one Mega is the documented upper bound, and it was already straining.//

===== Circuit =====

Passive components only (no op-amp):

  * 1N4148 diode (D1)
  * 10mH inductor (L1)
  * 0.1nF capacitor (C2)
  * 10nF capacitor (C1)
  * 10kΩ (R1) — Timer 1 / PWM pin 11 on Mega
  * 3.3kΩ (R2)
  * 1MΩ (R3) — Analog Pin 0

Send pin changes depending on Arduino model and which timer is used:
=== Mega ===
^ Timer ^ Pins set to OUTPUT ^
| 1 | 11, 12 |
| 3 | 2, 3, 5 |
| 4 | 6 |
| 5 | 46 |
=== Uno ===
^ Timer ^ Pin ^
| 1 | 9 (this is the "send" pin)|

//connect one leg of the 10nF cap to the electrode in the soil.//

==== Electrode material ====

The NIME paper's test used a "copper electrode roughly 5cm × 2cm", and a ~15cm rosemary plant with an electrode placed in soil. This is a documented working starting point.

===== Hardware — board compatibility =====

  * All Megas work unmodified: Original Mega (ATmega1280), Mega 2560. Mega ADK carries some risk with digital pin 7 so maybe stay away.
  * Mega 2560 has 256kb flash, 1280 only 128KB flash.
  * Uno vR1 through R3 works* (ATmega328P), **Uno R4 will not compile.**

**Architecture:** Mega runs SFCS → hardware UART (Serial1/2/3) → ESP32 → WiFi/OSC. UARTs have separate baud generators, no timer conflict. To avoid possible interrupt jitter, buffer readings, send in bursts between sweeps.

===== Multi-plant topology =====

One electrode per plant, shared soil, multiplexed. Shared soil causes crosstalk. Expect touched plant's channel to dominate, take strongest as winner..

The NIME paper's plant test found the soil disperses signal into the plant in a way that //widened// the range of CapSense results vs touching the electrode directly. Apparently soil coupling isn't purely a liability.

----

===== Reference links =====

[[https://www.nime.org/proceedings/2014/nime2014_515.pdf|NIME 2014 — Techniques in Swept Frequency Capacitive Sensing]] \\
The core paper. Includes circuit schematic with all component values including R\* as well as a side-by-side CapSense vs. SFCS test data on a rosemary plant. Also has the Cultivating Frequencies case study with its scaling limits. Primary source.

[[https://www.cultivatingfrequencies.com|Cultivating Frequencies]] \\
The installation's own site. The paper states they were collecting all data for the life of the piece and publishing it openly. The installation is at Aiguablava, Spain.
  * [[https://vimeo.com/85204158|installation]]
  * [[https://vimeo.com/258527723|proof of concept]]

[[https://s3-us-west-1.amazonaws.com/disneyresearch/wp-content/uploads/20140805145650/touchechi20121.pdf|Disney Research — Touché (2012)]] \\
The source technique. Custom hardware (AD5932 + ARM), no code released. Read for capability space and applications. Uses a pure sine wave.

[[http://dzlsevilgeniuslair.blogspot.se/2012/05/arduino-do-touche-dance.html|DZL's Evil Genius Lair — Arduino do the Touché dance]] \\
Nikolaj Møbius's original blog post — the actual origin of the Arduino circuit everything else derives from. Referenced by both the paper and damellis.

[[https://www.instructables.com/Touche-for-Arduino-Advanced-touch-sensing/|Instructables — Touché for Arduino]] \\
Mads Hobye's walkthrough, built on Møbius's work. Practical build guide and the code everything forked from.

[[https://github.com/chonigman/SweepingCapSense|GitHub — chonigman/SweepingCapSense]] \\
The library from the paper. Two objects: ''SweepingCap'' (drives the sweep) and ''Touch'' (stores per-frequency values — optional). Constructor takes either a timer number + frequency count, or just a count to use all four timers. Call ''setup()'' in setup, ''sweep(i)'' inside a for-loop over the frequency count. Dormant since ~2014.

[[https://github.com/damellis/ESP/wiki/%5BExample%5D-Touch%C3%A9-swept-frequency-capacitive-sensing|damellis/ESP wiki — Touché example]] \\
Circuit diagram with explicit pin assignments and build photos. Two sketches: ''Touche.ino'' (serial out) and ''Touche_LEDs.ino'' (receives classifications back, drives LEDs on pins 2/3/4). ML layer uses ''user_touche.cpp''; compile from source and uncomment the include in ''user.cpp''. Classifier algorithm not specified in the wiki :(

[[https://github.com/Surfincolin/plant-music|GitHub — Surfincolin/plant-music]] \\
The Pd/Ableton side is the useful part — reference for getting sensor data into a sound environment.

[[https://github.com/narner/Touche-Experiments|GitHub — narner/Touche-Experiments]] \\
Redundant and hard to parse. Archived 2020. ''Plants'' folder and a circuit photo, no standalone sensing code — built on damellis/ESP. Low priority.

[[https://medium.com/@narner/talking-to-plants-touch%C3%A9-experiments-1087e1f04eb1|Medium — Talking To Plants: Touché Experiments]] \\
Companion writeup to the above. Limited value.

----

===== Known risks =====

  * Plant-to-plant variability: some plants respond poorly or not at all. Disney never resolved which species work best
  * Soil moisture drift shifts baseline: may need periodic or automatic recalibration
  * The paper itself says long-run effects on data and interaction as the garden grows were unknown at publication
  * Grounding: one project reported needing the person coupled to circuit ground.

===== Alternative =====
  * **AD5932 sine wave generator:** what Disney used. Cleaner than our circuit but more complicated approach for starting out.
    * [[https://pmdway.com/products/ad9833-dds-programmable-frequency-function-generator|this is a good module for testing]]

===== Shelved =====
  * **AD5933 impedance spectroscopy:** two-electrode, measures impedance not self-capacitance. Possible module: KDT5933-013-SOU.
  * **Biopotential/electrophysiology:** more meaningful for passive environmental sensing, but every long-term result used lab-fabricated electrodes (SWCNT, thermoresponsive polymer, PEDOT:PSS). Off the table without university facilities