PComp Final Project — Aidan and Sophia

Project Title: Fruit Can Feel You Too

Project Description: Playing off the realities of artificial intelligence listening to us and connecting the digital and physical worlds together (i.e., phones listening and suggesting products on our Instagram pages), this project goes deeper, creating an experience for the digital world to connect with us through touch instead of sound.

Circuit:

Code:

Arduino Code

Page 1 of P5 Code

Page 2 of P5 Code

Page 3 of P5 Code

Process: We began this process by using an Adafruit documentation post for the Capacitive Touch Library. Unfortunately, the documentation was a little weak, so Sophia and I struggled at the start figuring out exactly how to get the Capacitive Touch Sensor working.

The Beginning of the Struggle

We had trouble getting the sensor working when we implemented fruit. We attached alligator clips to the sensor, but whenever the clip touched the sensor, it recognized a touch. We were wanting the touch to be sensed on the other side of the wire, on the fruit, but it wasn’t recognizing it because it was only recognizing the sensor board itself. We then troubleshooted, and, an hour later, reset the sensor and the Ardunio board and ran the code with the clips already attached to the sensor and the fruit, and it worked! When we touched the fruit, we got the number corresponding to which figure-8 the alligator clip was attached to on the sensor!

Once we were getting the identification number in the serial port in the Arduino app, we connected the Arduino to p5.js through the p5 serial communication app. Once in p5, we uploaded all of the songs we wanted to use in the project, and wrote a whole list of if-else statements telling p5 to open a certain page and play a certain song when it receives a certain number from Arduino. For example, whenever p5 received the number “4”, it was to open a new window on the computer using window.open() and play a song using watermelon.play(). Upon touch of the watermelon, an Amazon market page opened showing the landing page for purchasing watermelon, as well as played the song “Watermelon Sugar” by Harry Styles.

The process was pretty simple once we figured out how the board worked, and thanks to Sophia’s and my previous work with serial communication, we knew exactly the code we needed in Arduino and p5 to get the project working in its final state.

Happy moments/challenges:

Happy moments:

• Getting each of the phases of the process to work were huge victories themselves — it was exciting to see the sensor shield work once we soldered it, seeing the fruits trigger the serial monitor, getting the serial read to open a tab, and then getting songs to play in the p5 sketch.
• We also had a happy moment when we got to present and see everyone excited to trigger the outputs in class. After getting together a few times and working on this for many hours, seeing the interactions happen was very joyful

Challenges:

• We did not realize that we needed to attach the fruit to the shield before uploading the code. Without this, we were triggering the sensor while trying to hook everything up. We lost about 3 hours doing this, so that was awful
• We also faced the problem of not being able to put the fruit in a bowl since the fruits would trigger one another. The solution of placing them apart on the table ended up being fine, but we wish we could have presented them better
• We lastly faced a problem in our code that could have been changed in 3 seconds if we caught it sooner — serial read should have been serial write. Awesome.

If we had more time on this project, what would we improve or do differently?

Original Prototype of the Project

If we had more time, we would find a more enticing way of presenting the fruits to make people want to touch them — based on our prototype, we would’ve liked to have all fruits in one bowl together, but once they touched each other and someone touched one fruit, every sensor would trigger — then again, the need to touch things for the sake of touching them may drive people to touch the fruits anyway. We would also have spent more time on refining the code a bit — we got to a point where we could keep too many tabs from being opened at once, but there was no way for an already-touched fruit to return to it’s tab without us closing it. Nevertheless, we got done what we wanted to and it ended up being very satisfying and entertaining to interact with.

Where would a further developed iteration of this project be installed?

We can imagine this piece being in a gallery space where other art is being exhibited. Since this is a prototype, it would be interesting to have the different fruits and vegetables scattered around the room in different places as to emphasize the infiltration of technology on the more mundane elements of our lives. If you’re talking about cat food to a friend and you get an ad for cat food later in the afternoon on Instagram, we would want this piece to function in the same way. In a perfect world, the corresponding songs would be playing at a low volume as the onlookers observe other art in the space. The next day, they would receive some kind of email or message regarding the fruit they touched. This would require some kind of counter as well as some way of indicating people/differentiating them, but it would be a very cool application of this prototype.