During one of the most chaotic quarters of my undergrad - Autumn 2018 - I took [DXARTS 471: Mechatronic Art, Design, & Fabrication](https://dxarts.washington.edu/courses/2018/autumn/dxarts/471/a), taught by [Dr. Afroditi Psarra](https://afroditipsarra.com/). The class involved three main projects:
1. Media Archaeology
2. Speculative Object
3. Interface
These could all have been separate projects, but I decided to build on this theme that I've been wanting to explore.
# 1. Media Archaeology
The objective of this assignment was to re-imagine a piece of old technology into a new, expressive purpose. My parents had this old black-and-white CCTV in the garage, and I just knew I had to make it into a cute robot. It's rather simple at this stage: it's got a little pulsing heartbeat and just pans around the room.
# 2. Speculative Object
The objective of this assignment was to create something that would be from a world/ theme/ story of your imagination. Here's the lore that I'd come up with, building upon the cute little guy from the previous project:
> This piece envisions a future in which the issues of mass incarceration, global climate change, and the ethics of AI superimpose. AVL is a robot who was part of a farming robot colony, but malfunctioned and ended up destroying a few crops. AVL is extremely fond of plants - it’s all he cares about, but due to his glitches, he isn’t able to take care of them well. Due to the cost of repair, AVL was never fixed and was simply confined in a cage. Despite his good intentions, despite all the potential he had to sustain the dying world, he was just locked up and forgotten. Now, AVL simply watches whatever plants he can from within his cage. He monitors them, but he’s chained up and can’t water them. He’s grateful for the kind souls who water the plants, whether that be the clouds above, or a passerby. He’s far more grateful if that passerby shows empathy.
AVL pans between each plant and his heart displays one of three colors corresponding to the soil moisture level. If a viewer waters a plant, AVL switches to looking at that plant and his heart rapidly beats blue.
# 3. Interface
The objective of this assignment was to create an interface! Here's what I'd written about it:
> The goal of the piece is to take empathy to its furthest extent: putting you inside the robot software’s perspective, and fixing the robot’s "glitched" software through empathy. Ever since I began practicing fundamental engineering principles, I’ve always believed that empathy is the basis of every good solution. Complex problem solving requires true democracy - understanding the perspectives of multiple stakeholders. An immersive VR interface like this one does its best to simulate one such perspective. The impact is amplified by the cage - in real life - breaking open when the user opens it in VR.
# Technical Details
## Functional Description
The head is an old black and white TV, which connects to a camera mounted on the top of the cage. The camera direction is controlled by a servo motor. There’s a soil moisture sensor in each plant. The camera pans from plant to plant, “scanning” each one (reading the moisture levels) and outputting its status (red = dry, yellow = good, green = currently being watered) via the LED behind the heart. The moment a plant is being watered, the camera will look at it and express joy (through the LED flashing blue). The code replaces delays with multithreading.
The VR interface is implemented by a Microsoft Mixed Reality headset, a VR-ready laptop, Unity, and the Mixed Reality Toolkit for Unity. The hardware entails a USB Serial connection between Unity and an Arduino - which controls a servo motor. After the game asks the user to complete tasks to fix the robot, the game instructs the user to open the virtual cage by touching a handle on the back gate; when that handle is touched, a message is sent to the Arduino that the cage should be opened, and the Arduino turns the servo, popping the cage open.
## Circuit Schematic
![[specobj_bb.png|circuit schematic]]
## Code
Kudos to the [Microsoft Mixed Reality Toolkit for Unity](https://github.com/Microsoft/MixedRealityToolkit-Unity), and various free non-commercial 3D models online.
### Opening the Cage (Arduino):
```C++
#include <Servo.h>
Servo myservo;
int pos = 0; // position of servo
int data;
void setup() {
Serial.begin(9600);
pinMode(13, OUTPUT);
myservo.attach(6);
myservo.write(1); // reset the cage to closed
}
void loop() {
if (Serial.available()) {
data = Serial.read();
if (data == 'A') {
digitalWrite(13, HIGH);
myservo.write(90); // open the cage
} else {
digitalWrite(13, LOW);
}
}
}
```
### Opening the Cage (Unity):
```Java
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.IO.Ports;
public class collide_test : MonoBehaviour {
public SerialPort serial = new SerialPort("COM5", 9600);
private bool rotateBool = true;
private void OnTriggerEnter(Collider other)
{
if (serial.IsOpen == false)
{
serial.Open();
}
if (rotateBool)
{
serial.Write("A");
rotateBool = false;
}
else
{
serial.Write("B");
rotateBool = true;
}
}
}
```
### Plant Animation Trigger (Unity):
(there are a few other small animations like this)
```Java
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public class animController : MonoBehaviour {
public Animator anim;
private void OnTriggerEnter(Collider other)
{
anim.Play("plant-1-animation");
Debug.Log("Seed planted!");
cageScript.plantCount++;
}
}
```
### Main Robot
```C++
/*
* The goal of this code is to pan the servo motor whilst reading
* sensor data, and react to that sensor data ASAP (as if they were
* occurring in parallel). This is done by substituting the delay between
* servo turns with sensor checks.
*
* We're checking the status of three water sensors by panning to each one.
* Meanwhile, if any one of them becomes wet, the camera will react by looking
* at that plant for a short duration and pulsing the LED blue.
*
*/
#include <Servo.h>
Servo myservo;
int pos = 0; // position of servo
// sensor IDs of the plants
const int S1 = A0;
const int S2 = A1;
const int S3 = A2;
// angle values of the plants
const int P1 = 58;
const int P2 = 83;
const int P3 = 110;
// soil moisture sensor thresholds
const int WATER_THRESH = 750;
const int HUMID_THRESH = 500;
const int DRY_THRESH = 300;
// LEDs
const int R = 9;
const int G = 10;
const int Y = 11;
void setup() {
Serial.begin(9600);
myservo.attach(6);
pinMode(R, OUTPUT);
pinMode(G, OUTPUT);
pinMode(Y, OUTPUT);
}
void loop() {
pan(P1, S1, P2);
pan(P2, S2, P3);
pan(P3, S3, P2);
pan(P2, S2, P1);
}
void pan(int plant1, int sensor1, int plant2) {
scan(plant1, sensor1);
// pan from plant1 to plant 2
if (plant2 > plant1) {
for (int i = plant1; i <= plant2; i++) {
myservo.write(i);
check();
}
} else {
for (int i = plant1; i >= plant2; i--) {
myservo.write(i);
check();
}
}
}
/* scans a single plant for approximately a second
* whilst panning
*/
void scan(int plant, int sensor) {
myservo.write(plant);
for (int i = 0; i <= 50; i++) {
check();
status(sensor);
}
}
/*
* Takes a little over 30ms to accomplish this
*
* checks all the plants for watering
*/
void check() {
for (int i = 0; i <= 100; i++) {
checkPlant(P1, S1);
checkPlant(P2, S2);
checkPlant(P3, S3);
}
}
// checks if an individual plant is
// being watered
void checkPlant(int plant, int sensor) {
if (analogRead(sensor) > WATER_THRESH) {
react(plant, G);
}
}
// reacts to a plant being watered
void react(int plant, int reactLED) {
resetLEDs();
myservo.write(plant);
delay(1000);
for (int i = 0; i < 5; i++) {
Serial.println("Water!!!!");
digitalWrite(reactLED, HIGH);
}
delay(2000);
digitalWrite(reactLED, LOW);
}
// reports the status of the given plant
// with LED
void status(int sensor) {
resetLEDs();
if (analogRead(sensor) > HUMID_THRESH && analogRead(sensor) <= WATER_THRESH) {
digitalWrite(Y, HIGH);
} else {
digitalWrite(R, HIGH);
}
}
void resetLEDs() {
digitalWrite(R, LOW);
digitalWrite(G, LOW);
digitalWrite(Y, LOW);
}
```
### Calibration Code - Servo Motor Angle (Arduino):
```C++
/*
* Helps callibrate the camera angle
*/
#include <Servo.h>
Servo myservo;
int pos = 0; // position of servo
void setup() {
Serial.begin(9600);
myservo.attach(6);
}
void loop() {
int angle = Serial.parseInt();
if (angle != 0) {
myservo.write(angle);
}
```
### Calibration Code - Water Sensor Check (Arduino):
```C++
/*
MODIFIED BY: Anand Sekar (11/12/18)
# Example code for the moisture sensor
# Editor : Lauren
# Date : 13.01.2012
# Version : 1.0
# Connect the sensor to the A0(Analog 0) pin on the Arduino board
# the sensor value description
# 0 ~300 dry soil
# 300~700 humid soil
# 700~950 in water
*/
void setup(){
Serial.begin(57600);
}
void loop(){
Serial.print("MSV 1:");
Serial.println(analogRead(A0));
Serial.print("MSV 2:");
Serial.println(analogRead(A1));
Serial.print("MSV 3:");
Serial.println(analogRead(A2));
delay(100);
}
```