Testing sensor with those that brought them ( 15 min )
Part 1: FILTERS — look at a range of filters (maybe an hour)
Part 2: FEATURES — some simple thresholding and edge detection once filtered (30-40)
questions, project ideas, case studies, ?? leave ???
BEFORE we meet:
BUILD: if you want to use hardware ( i hope you do — it would be fun to have many people exploring), try to build your analog sensor circuit and test it before we meet
you only need one sensor for the session — photocell would be fine.
i will demo a simple photocell, a potentiometer and a sonar sensor — just to give you a sense of how different sensors react
the code only needs to know the Analog Input you are using.
if you have an SPI or I2C sensor — you should get sensor specific libraries in advance and modify code —> getSingleSensorReading() (see comments in code)
This algorithm stores the sum used in averaging in a variable — avoiding need to constantly (re)iterate the array to get new average.
This method also uses module (%) operator to simplify the iteration.
In practice % takes an incrementing variable (INDEX) and constrains it loop over a range of your choice ( windowSize )
<confusion warning — technical doesn’t always help> — mathematically % returns the numerator of the remainder of a division (huh?)
ASIDE : Using Modolo (%)
code sample
+ very effective
+ uses RECENT data, user defined size
- fancy code may be confusing — but its worth digging into
- needs an array to hold old values — can be memory intensive if big window
- not a factor in this implementation but large windows can slow down your code if you have to iterate for the sum
= sensitivity is adjusted by changing window size — big window, stronger averaging impact (but at speed cost)
Exponential Moving Average
one of my favourites — proportionally mixes new readings with old readings
I am using a float version from 0.0-1.0 ( INT versions are faster but floats are a good start — they make it easier to see what is happening)
Procedure
define proportion or weight for new readings —> called EMA_weight in code
proportion for OLD readings will be —> 1.0 - EMA_weight , because the two proportions have to add up to 1.0
get a new sensor reading — getSingleSensorReading()
calculate —> EMA = ( EMA_weight * newReading ) + ( 1- EMA_weight * EMA )
that’s it
(note EMA variable is used in last term! — sneaky)
What’s Happening Here?
EMA is made up of two proportional terms (new + old):
New reading term is first—> ( EMA_weight * newReading )
Old readings terms is second —> it has two steps
fist calculate relative weight of ALL OLDER readings — ( 1.0-EMA_weight)
multiplying the previous EMA by this weight to get proportional value of all old readings
add the new reading term to the old reading term
IF weight is close to 1.0 —> newReadings are more important than old readings (in fact weight=1) is just the newReading
IF weights is close to 0.0 —> old Readings are more important than new readings (zero doesn’t actually work but 0.001 will give almost ALL old and tiny nudge from new readings)
Find big GREEN CODE button — select bottom item —> Download ZIP
Move ZIP to desktop ( if its not there already ).
In ARDUINO software, goto Sketch—> IncludeLibrary—> Add.ZIP library
Navigate to ZIP you just downloaded and CHOOSE it.
Wait for Arduino to do its thing.
CONFIRM INSTALL: Arduino—>File—> Examples—> Scroll down and look for MedianFilter-master. If its on the list (it will be way down) then its installed.
Procedure
most of the heavy lifting is done by the library::
you need to include library —> #include <MedianFilter.h>
you need a window size — like moving average —> medianWindowSize = 16;
instantiate the object, defining window and seed —> MedianFilter median_readings( medianWindowSize, seed );
(note — seed is a guess s to what the median may be — i set mine to 0 -- may change first few medians — not sure)
once all setup, use library .in() to add new readings and .out() to get the median
median_readings.in( newReading ); // adds new reading to window
sensorMedian = median_readings.out();
What’s happening here?
MEDIAN is calculated by holding data in array (much like we did in moving average)
The array is SORTED small to big
The value stored at the MIDDLE INDEX of the sorted array is the median.
because the data is sorted — spikes (up or down) get sorted to the ends of the storage array - when we pick the middle value, we avoid the spike
Notes
- lags can be noticeable
- because of sorting — VERY sensitive to window size — big windows = big lags
+ library handles everything — yay libraries
+ excellent spike removal
sensors like loudness detectors (mics) are seeking spikes — so avoid median filters in these cases
Flattener (notch filter)
i just call this a flattener — it probably has a better name
a great way to get rid of “flicker” in super stable sensors (eg. Potentiometers)
AKA - if it didn’t change a lot — it didn’t change at all
Procedure
determine your TOLERANCE for a change that matters (SMALLer == more sensitive)
SAMLL is relative here but I expect 1-20 points on the analog read scale ( 0-1024 )
get a sensorReading and STORE it for later, eg —> oldSensorValue
get new sensorReading and subtract it from oldSensorValue (the one we just stored)
get the absolute of the difference we just calculated ( we only care of about size of difference )
if the ABSOLUTE DIFFERENCE is SMALL ( less than (<) TOLERANCE )— use the OLD reading in place of the NEW reading
if the ABSOLUTE DIFFERENCE is BIG ( greater than (>) TOLERANCE ) — use the NEW reading.
Notes
some sensors are rock solid — and you can actually select values that sit between Arduino levels — this results in flicker — flattener to the rescue
this can be misused as the lazy solution to all noise — avoid brute forcing things when looking for subtlety.
this filter crosses over into feature detection
Part 2: Features
Checking for Change
ASIDE: Using timers
delay(someTime); is a good tool if blocking code execution doesn’t matter.
when you need to do have events happen periodically, but do not want to block your code, then timers area better option
with timers, you define an interval between events, check to see if that interval has passed and reset the timer.
timers are easy to set up though they take more code than delay.
basic timer structure
unsigned long timeNow;
unsigned long eventInterval = 50; // in ms
unsigned long lastEventTime = 0;
timeNow = millis();
if (( timeNow - lastEventTime) > eventInterval ){
// event tasks happen here
lastEventTime = timeNow; // reset timer
}
Thresholds
thresholds are user defined sensor levels that can be used to determine different kinds of actions
determining what level is best for a threshold is context dependant. The following can all impact setting / determining a threshold.
the staging of the sensor
the type of sensor
the environment of the sensor
user actions
Procedure:
For a simple threshold — make note of typical sensor readings in two distinct situations (lights ON vs OFF — or — user CLOSE vs FAR)
Set your threshold to a value near the midpoint between those levels (could be average, a median or just an estimate — i usually guess at first)
Get a new sensor reading
Check to see if new reading is above or below the threshold
Take appropriate action for each state.
Notes:
you can use more than one threshold at time too create a staircase-like set of levels
in such a case, the stability of your sensor influences how close together different thresholds can be
a noisy sensor will need wider bands — or greater difference between adjacent thresholds
actions can take any form once triggered
Change Detection
once you have a threshold set we can ask some questions about our system in relation to it.
a basic one is has the sensor changed?
Procedure:
set up threshold
capture a new reading (filter it as needed)
compare newReading (this loop) to old Reading ( from last loop )
if ( new!=old ), state has changed
** save newReading AS oldReading to use for comparison in next loop
Notes:
you can check for change in filtered values or threshold values
remember to save the values from the current loop to enable comparison in later loops
this can be used with digital sensors too — can reduce amount of messaging you get from a sensor
Edge Detection
Once you have threshold determined you may want to detect direction of change in your sensor
We may call a sensor reading above threshold = HIGH, and one below = LOW
We then call RAISING edges those that transition from LOW to HIGH
and we call FALLING edge those that transition from HIGH to LOW
Procedure:
set up threshold
capture a new reading (filter it as needed)
compare newReading (this loop) to old Reading ( from last loop )
if ( newReading > oldReading ) == RAISING
if ( oldReading < newReading ) == FALLING
** save newReading AS oldReading to use for comparison in next loop
Notes:
the transition can be very fast or slow
if you compare threshold values the transition will fire only once or close to once.
if you compare filtered readings, you may see transitions over several loops as filtering can introduce short time lags.
Extras
I will have couple of visualization demos that use P5JS — you can try them out if you like — you will need P5.serialcontrol to make it all happen. Find releases here: https://github.com/p5-serial/p5.serialcontrol/releases
I think I am using latest? BETA 1.2 should do the trick.
