Class Notes:

Homework:

• [ ] Choose any model available in Transformers.js, such as:
• Object Detection: 🕵️ Detect objects in an image.
• Image Segmentation: ✂️ Segments an image into parts.
• Text Generation: ✍️ Generate text based on a user prompt.
• Whisper (Speech-to-Text): 🎙️ Transcribes audio from a file or real-time microphone.
• [ ] Compare and Contrast:
  • Compare your experience of using Transformers.js with ml5.js. Reflect on differences in ease of use, performance, model variety, and integration with p5.js.
• [ ] Document your project in a blog post and add a link to your documentation to the Assignment Wiki page. Remember to include visual documentation (screenshots, GIFs, etc) Also, please document what did and did not work! What errors did you encounter with Transformers.js?

Brainstorm + Ideas:

1. Using the depth sensor create a simulation of what is it like to be far sighted on a scale of 10. When input is entered you get to see the perspective of a person and how far sighted they are.
   • Since, using transformers tends to take long and have difficulty loading and my programming skills aren’t the best I decided to choose a simple project to complete the task.
   • A bonus if time, add a pixelization feature.
2. https://huggingface.co/rhymes-ai/Aria This was another interesting transformers where you could use with Rhymes. So perhaps, a karaoke style finish the rhyme. Or a poetry helper/assiter.

Test for Eye Vision Test:

• Code #1:
• Code #2:

While testing to see how I can change the level of depth in an image, the web editor would crash at time…

Sometimes the web editor would not load and lead to an error?

I think for this project I tried to keep it simples due to the wait time and website glitching once in a while.

https://editor.p5js.org/TamaraBueno/sketches/sn_JeH5Gk

So then I decided to try again using the ‘canyon’ image.

Test for Ryhmes:

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("rhymes-ai/Aria", trust_remote_code=True)

I am unsure as to how to load the rhymes file into p5.js…. as I kept receiving errors.

https://editor.p5js.org/